in find_spec_for_exe: can't find gem bundler

posted 2020-06-12, filed under "hell is other people's code", "debugging"

Hell is other people’s code.
~ Jean-Paul Sartre, 2020

I just made a few updates to my site, which involved moving some pages and adding redirects from the old URLs. Since this is a Jekyll site the obvious choice is jekyll-redirect-from, which seemed to work fine. But then when I went to try to deploy the site to my cheap ass DigitalOcean server, all hell broke loose.

tbodt@tbodt ~/r/tbodt.com> bundle install
Traceback (most recent call last):
	2: from /usr/local/bin/bundle:23:in `<main>'
	1: from /usr/lib/ruby/2.5.0/rubygems.rb:308:in `activate_bin_path'
/usr/lib/ruby/2.5.0/rubygems.rb:289:in `find_spec_for_exe': can't find gem bundler (>= 0.a) with executable bundle (Gem::GemNotFoundException)

OK, so I have to install bundler.

tbodt@tbodt ~/r/tbodt.com> gem install bundler
ERROR:  While executing gem ... (Gem::FilePermissionError)
    You don't have write permissions for the /var/lib/gems/2.5.0 directory.
tbodt@tbodt ~/r/tbodt.com> sudo gem install bundler   # fvck
Fetching: bundler-2.1.4.gem (100%)
Successfully installed bundler-2.1.4
Parsing documentation for bundler-2.1.4
Installing ri documentation for bundler-2.1.4
Done installing documentation for bundler after 4 seconds
1 gem installed
tbodt@tbodt ~/r/tbodt.com> bundler
Traceback (most recent call last):
	2: from /usr/local/bin/bundler:23:in `<main>'
	1: from /usr/lib/ruby/2.5.0/rubygems.rb:308:in `activate_bin_path'
/usr/lib/ruby/2.5.0/rubygems.rb:289:in `find_spec_for_exe': can't find gem bundler (>= 0.a) with executable bundler (Gem::GemNotFoundException)
tbodt@tbodt ~/r/tbodt.com> sudo gem uninstall bundler
Gem 'bundler' is not installed

WTF??

Most people would immediately start uninstalling and reinstalling things randomly in the hope that something would happen. I tried that too, several times, but it did fuckall, and even led me down the wrong path of thinking it was installing the gems in /usr/lib/ruby/gems and looking for them in /var/lib/gems. So now I enter the fires of hell. Can we at least load the gem from IRB?

tbodt@tbodt ~/r/tbodt.com> irb
irb(main):001:0> gem 'bundler'
Traceback (most recent call last):
        5: from /usr/bin/irb:11:in `<main>'
        4: from (irb):1
        3: from /usr/lib/ruby/2.5.0/rubygems/core_ext/kernel_gem.rb:65:in `gem'
        2: from /usr/lib/ruby/2.5.0/rubygems/dependency.rb:322:in `to_spec'
        1: from /usr/lib/ruby/2.5.0/rubygems/dependency.rb:312:in `to_specs'
Gem::MissingSpecVersionError (Gem::MissingSpecVersionError)

<debugging instinct kicks in> OK, we have a stack trace. Let’s narrow it down and then look at the code.

irb(main):002:0> Gem::Dependency.new('bundler')
=> <Gem::Dependency type=:runtime name="bundler" requirements=">= 0">
irb(main):003:0> Gem::Dependency.new('bundler').to_spec
Traceback (most recent call last):
        4: from /usr/bin/irb:11:in `<main>'
        3: from (irb):3
        2: from /usr/lib/ruby/2.5.0/rubygems/dependency.rb:322:in `to_spec'
        1: from /usr/lib/ruby/2.5.0/rubygems/dependency.rb:312:in `to_specs'
Gem::MissingSpecVersionError (Gem::MissingSpecVersionError)

rubygems/lib/rubygems/dependency.rb:382:

class Gem::Dependency
  def to_specs
    matches = matching_specs true
    # ...
  end

  def matching_specs(platform_only = false)
    env_req = Gem.env_requirement(name)
    matches = Gem::Specification.stubs_for(name).find_all do |spec|
      requirement.satisfied_by?(spec.version) && env_req.satisfied_by?(spec.version)
    end.map(&:to_spec)

    Gem::BundlerVersionFinder.filter!(matches) if name == "bundler".freeze && !requirement.specific?

    if platform_only
      matches.reject! do |spec|
        spec.nil? || !Gem::Platform.match(spec.platform)
      end
    end

    matches
  end
end

This has several tests for which “stubs” satisfy all the right requirements.

irb(main):001:0> Gem::env_requirement('bundler')
=> #<Gem::Requirement:0x000055a1400b40f0 @requirements=[[">=", #<Gem::Version "0">]]>
irb(main):005:0> Gem::Dependency.new('bundler').requirement
=> #<Gem::Requirement:0x0000555630ae8db0 @requirements=[[">=", #<Gem::Version "0">]]>
irb(main):008:0> Gem::Specification.stubs_for('bundler')[1]
=> nil
irb(main):009:0> Gem::Specification.stubs_for('bundler')[0].version
=> #<Gem::Version "2.1.4">
irb(main):010:0> Gem::Dependency.new('bundler').requirement.satisfied_by?(Gem::Specification.stubs_for('bundler')[0].version)
=> true

Looks like there’s only one “stub” (I still don’t know what that is) for bundler, and it satisfies the very loose requirement that the version must be at least 0. So then why is it failing? Hmm, that BundlerVersionFinder thing looks suspicious, especially since it only applies if you are asking for a gem called “bundler”…

module Gem::BundlerVersionFinder
  def self.filter!(specs)
    return unless bundler_version = self.bundler_version

    specs.reject! {|spec| spec.version.segments.first != bundler_version.segments.first }

    exact_match_index = specs.find_index {|spec| spec.version == bundler_version }
    return unless exact_match_index

    specs.unshift(specs.delete_at(exact_match_index))
  end
end

Looks like it’s trying to ignore any specs that are the “wrong” version of bundler, i.e. self.bundler_version. So what’s the “right” version?

irb(main):015:0> Gem::BundlerVersionFinder.bundler_version
=> #<Gem::Version "1.16.1">

Huh, I’m pretty sure I just installed bundler 2.1.4. Why does it want this version specifically?

module Gem::BundlerVersionFinder
  def self.bundler_version
    version, _ = bundler_version_with_reason
    return unless version
    Gem::Version.new(version)
  end

  def self.bundler_version_with_reason
    if v = ENV["BUNDLER_VERSION"]
      return [v, "`$BUNDLER_VERSION`"]
    end
    if v = bundle_update_bundler_version
      return if v == true
      return [v, "`bundle update --bundler`"]
    end
    v, lockfile = lockfile_version
    if v
      return [v, "your #{lockfile}"]
    end
  end
end
irb(main):001:0> Gem::BundlerVersionFinder.bundle_update_bundler_version
Traceback (most recent call last):
        2: from /usr/bin/irb:11:in `<main>'
        1: from (irb):1
NoMethodError (private method `bundle_update_bundler_version' called for Gem::BundlerVersionFinder:Module)
Did you mean?  bundler_version_with_reason

Fuck you. Wait, isn’t Ruby like Objective-C, in that access control is only a suggestion?

irb(main):002:0> Gem::BundlerVersionFinder.send :bundle_update_bundler_version
=> nil
irb(main):003:0> Gem::BundlerVersionFinder.lockfile_version
Traceback (most recent call last):
        2: from /usr/bin/irb:11:in `<main>'
        1: from (irb):3
NoMethodError (private method `lockfile_version' called for Gem::BundlerVersionFinder:Module)
irb(main):004:0> Gem::BundlerVersionFinder.send :lockfile_version
=> ["1.16.1", "/home/tbodt/repos/tbodt.com/Gemfile.lock"]

Ah fuck it’s the lockfile.

tbodt@tbodt ~/r/tbodt.com> nvim Gemfile.lock

Sure enough, at the bottom:

BUNDLED WITH
   1.16.2

Well now that I know what the problem is, it’s easy to fix:

tbodt@tbodt ~/r/tbodt.com> gem install bundler@1.16.1
ERROR:  Could not find a valid gem 'bundler@1.16.1' (>= 0) in any repository
tbodt@tbodt ~/r/tbodt.com> sudo gem install bundler -v 1.16.1
Fetching: bundler-1.16.1.gem (100%)
Successfully installed bundler-1.16.1
Parsing documentation for bundler-1.16.1
Installing ri documentation for bundler-1.16.1
Done installing documentation for bundler after 3 seconds
1 gem installed
tbodt@tbodt ~/r/tbodt.com> bundle install
Fetching gem metadata from https://rubygems.org/...........

But this was way way too hard to figure out. Remember, the original error was can't find gem bundler (>= 0.a) with executable bundle (Gem::GemNotFoundException). Who the hell looks at that and goes “oh of course, my lockfile version is mismatched!?” I had to go and spend an hour debugging other people’s code, the purest form of hell. And this sort of thing happens all the time! At least this time I was able to turn it into a blog post.