The immortal symbols

TL; DR; Always validate the names of the supported methods when we perform metaprogramming.

In ruby, it is very common to use symbols for almost everything, since they are more efficient when making searches or comparisons than simple strings. And thanks to the garbage collector we don't worry about the memory allocations that our symbols use. However, there is a scenario where the garbage collector ignores them and enables "immortal symbols". Which will never be eliminated by the garbage collector and will live forever in memory.

Let's do a little test, we will use ObjectSpace for practical purposes and in this case, we will only focus on symbols and strings:

require 'objspace'
require 'SecureRandom'

puts ObjectSpace.count_objects.slice(:T_SYMBOL, :T_STRING)

10.times { SecureRandom.hex.to_sym }

puts ObjectSpace.count_objects.slice(:T_SYMBOL, :T_STRING)

GC.start

puts ObjectSpace.count_objects.slice(:T_SYMBOL, :T_STRING)

In the execution we have the following result:

$ ruby inmortal.rb
{:T_SYMBOL=>28, :T_STRING=>10107}
{:T_SYMBOL=>38, :T_STRING=>10179}
{:T_SYMBOL=>28, :T_STRING=>7787}

As we can see at the beginning, we have several strings and symbols. When we call SecureRandom.hex, we instantiate a string and then it is converted into a symbol. Since none is used, the garbage collector removes these symbols and some other strings.

Now, suppose we have some metaprogramming in our code, for which we will use method_missing, send and define_method

require 'objspace'
require 'SecureRandom'

puts ObjectSpace.count_objects.slice(:T_SYMBOL, :T_STRING)

class Inmortal
  def method_missing(method, *args, &block)
    create_method(method)
    send(method)
  end

  def create_method(name)
    self.class.define_method(name) { "method_#{name}/0" }
  end
end

my_inmortal = Inmortal.new

10.times { my_inmortal.send(SecureRandom.hex) }

puts ObjectSpace.count_objects.slice(:T_SYMBOL, :T_STRING)

GC.start

puts ObjectSpace.count_objects.slice(:T_SYMBOL, :T_STRING)

And we have as a result:

$ ruby inmortal.rb
{:T_SYMBOL=>28, :T_STRING=>10136}
{:T_SYMBOL=>38, :T_STRING=>10228}
{:T_SYMBOL=>38, :T_STRING=>7793}

What happened here? When we generated 10 new symbols, we never did a cast to a symbol. To do this, we will debug method_missing by adding:

   def method_missing(method, *args, &block)
    puts method.class
    create_method(method)
    send(method)
  end

and now this happens:

{:T_SYMBOL=>28, :T_STRING=>10135}
Symbol
Symbol
Symbol
Symbol
Symbol
Symbol
Symbol
Symbol
Symbol
Symbol
{:T_SYMBOL=>38, :T_STRING=>10144}
{:T_SYMBOL=>38, :T_STRING=>7793}

Ok, apparently method_missing is receiving a symbol, but who is casting and sending it? let's investigate inside the method send:

  def send(method, *args, &block)
    puts method.class
    super
  end

and we got:

$ ruby inmortal.rb
{:T_SYMBOL=>28, :T_STRING=>10152}
String
Symbol
String
Symbol
String
Symbol
String
Symbol
String
Symbol
String
Symbol
String
Symbol
String
Symbol
String
Symbol
String
Symbol
{:T_SYMBOL=>38, :T_STRING=>10164}
{:T_SYMBOL=>38, :T_STRING=>7794}

So the method send is being called twice per method, the first time as a string then calls method missing with the cast to symbol

Invokes the method identified by symbol, passing it any arguments specified. When the method is identified by a string, the string is converted to a symbol.

According to the documentation send cast to symbol.

Ok, we already determine who converts to symbols, but why the garbage collector never removes those symbols?

Lets deep dive, If we modify our benchmark to see the methods defined in our class we will have some insights

puts my_inmortal.methods.count
10.times { my_inmortal.send(SecureRandom.hex) }
puts my_inmortal.methods.count

and we will get:

$ ruby inmortal.rb
{:T_SYMBOL=>28, :T_STRING=>10134}
60
70
{:T_SYMBOL=>38, :T_STRING=>10228}
{:T_SYMBOL=>38, :T_STRING=>7793}

As we can see, our class instance has 10 new method definitions which are symbols, therefore the garbage collector ignores those references since they belong to an object instance. Even if the object instance is removed entirely, the reference to that object's methods lives forever.

So how can we avoid these memory leaks? By just avoiding unwanted methods creation:

require 'objspace'
require 'SecureRandom'

puts ObjectSpace.count_objects.slice(:T_SYMBOL, :T_STRING)

class Inmortal
  def method_missing(method, *args, &block)
    super unless method.start_with?('m_')
    create_method(method)
    send(method)
  end

  def respond_to_missing?(method, include_private = false)
    method.start_with?('m_') or super
  end

  def create_method(name)
    self.class.define_method(name) { "method_#{name}/0" }
  end
end

my_inmortal = Inmortal.new

10.times { my_inmortal.send("m_#{SecureRandom.hex}") }
10.times { my_inmortal.send(SecureRandom.hex) rescue nil }

puts ObjectSpace.count_objects.slice(:T_SYMBOL, :T_STRING)

GC.start

puts ObjectSpace.count_objects.slice(:T_SYMBOL, :T_STRING)

Resulting:

$ruby inmortal.rb
{:T_SYMBOL=>28, :T_STRING=>10145}
{:T_SYMBOL=>48, :T_STRING=>10204}
{:T_SYMBOL=>38, :T_STRING=>7805}

As we can see, when we validate both in "respond_to_missing" and in "method_missing" the creation of a new method is avoided, therefore there will be no immortal symbols, then just the required for the instance.

Basically you could make a memory bomb in ruby by doing something like this:

require 'SecureRandom'

class Bomb
  def method_missing(meth, *args, &blk)
    self.class.send(:define_method, "is_#{meth}?") { true }
    send("is_#{meth}?")
  end
end

loop { Bomb.new.send(SecureRandom.hex) }

So now you know, be careful with your symbols when you do metaprogramming.