CocoaPods is so popular it’s straining GitHub’s servers

CocoaPods, one of the most popular ways to integrate libraries and dependencies into apps, recently had some performance issues. As it turns out, the reason was that it was too popular.

Developers who were trying to install Pods may have had it timeout a few times, or just plain not work. An issue listing on the CocoaPods GitHub repo has since seen a response from GitHub employees, who note CocoaPods basically took down five servers.

From user mhagger, an engineer on GitHub’s infrastructure team:

The slow fetches and clones (which sometimes time out) that the CocoaPods community is experiencing are caused by automatic rate limiting on our servers, which is done to make sure that extremely high levels of load in one repository cannot impact other GitHub users. The CocoaPods/Specs repository is more or less permanently being rate limited.

If you’re wondering why CocoaPods repos are being rate limited, mhagger says it’s basically due to popularity:

There have been approximately 1.1 Million clones/fetches from CocoaPods/Specs in the past week. This activity has kept, on average, more than 5 server CPUs permanently pegged, and used several terabytes of bandwidth out of our datacenters. There are only a handful of other repositories in all of GitHub that even come close to this level of activity. As far as I know, this level of activity is not new, but has been going on for many months and probably longer. Suffice it to say that the name CocoaPods/Specs is quite well known within our team.

GitHub will fix these issues, so CocoaPods hasn’t hit some kind of glass ceiling. The team at GitHub is also encouraging CocoaPods to change the way it accesses repo clones (full rather than shallow), which will decrease server load and help avoid these access point problems in the future.

➤ Issues Cloning Spec repo – GitHub taking a very long time to download changes to the Specs Repo [GitHub]