Tuesday, June 06, 2017

One of the configured repositories failed (Unknown), and yum doesn't have enough cached data to continue.

Running AWS Minimal Linux instance got this error in the logs on start up:

Starting cloud-init: Cloud-init v. 0.7.6 running 'modules:config' at Tue, 06 Jun 2017 09:10:02 +0000. Up 18.86 seconds.
Loaded plugins: priorities, update-motd, upgrade-helper

 One of the configured repositories failed (Unknown),
 and yum doesn't have enough cached data to continue. At this point the only
 safe thing yum can do is fail. There are a few ways to work "fix" this:

     1. Contact the upstream for the repository and get them to fix the problem.

     2. Reconfigure the baseurl/etc. for the repository, to point to a working
        upstream. This is most often useful if you are using a newer
        distribution release than is supported by the repository (and the
        packages for the previous distribution release still work).

     3. Disable the repository, so yum won't use it by default. Yum will then
        just ignore the repository until you permanently enable it again or use
        --enablerepo for temporary usage:

            yum-config-manager --disable <repoid>

     4. Configure the failing repository to be skipped, if it is unavailable.
        Note that yum will try to contact the repo. when it runs most commands,
        so will have to try and fail each time (and thus. yum will be be much
        slower). If it is a very temporary problem though, this is often a nice
        compromise:

            yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true


There's a lot of information there but if you scroll down a bit farther you'll see something like this:

Could not retrieve mirrorlist http://repo.us-west-2.amazonaws.com/latest/main/mirror.listerror was
12: Timeout on http://repo.us-west-2.amazonaws.com/latest/main/mirror.list: (28, 'Connection timed out after 10001 milliseconds')

This means the network configuration is blocking access the Amazon Yum Repo. 

When I look up the IP address associated with the above repo I get at this moment: 

nslookup repo.us-west-2.amazonaws.com

Name: s3-us-west-2-w.amazonaws.com
Address: 52.218.144.34

OK so apparently the AWS Yum repositories are hosted on S3.

We can look up the latest AWS IP Ranges here:

AWS IP Range Updates Are Published As Explained Here:

There could be a few things causing this:

1. Security group rules disallow access to S3 IP ranges.
2. NACLs do not allow access to S3 IP ranges
3. No Internet Gateway route in subnet route table
4. The traffic routes from this subnet to another subnet in or outside AWS that can access the Internet, but traffic is blocked by a proxy, NAT or firewall that is disallowing the traffic.

As noted using a NAT on AWS is challenging but can be used to route traffic for updates from instances in a private network to the public Internet.


It may also be a good idea to internally host software on a repository that does not traverse the Internet to get updates. Consider Nexus, Artifactory or similar solutions. Then reaching out to the Internet would be limited to a few computers used to run the repository. A private repository could also be hosted on S3 with an S3 Endpoint.