Always stop unused Akka actors

Recently I came across an easy but interesting problem. An Akka-based application was failing because of an OutOfMemoryError. It was a typical memory leak.

The class is an Akka actor that works like this:

  • a new instance of the actor is created when a file needs to be downloaded and the actor receives a message which tells it to download the file
  • the actor stores the reference to the sender
  • it sends an HTTP request to a service
  • responses are delivered as messages to the same actor
  • the actor can receive either the complete response or a chunk. If it has received a chunk the actor keeps storing the data in a byte array until the whole file is delivered
  • finally, when the actor has the entire response, it sends it to the original caller

It can handle only one request, and it is useless after returning the content of the file. That is not a problem. The problem is that it keeps the data in memory even after sending the response back! There are no references to the actor in the application code, but there still is a parent-child relationship defined in the actor system. The actor still exists!

How can I remove an actor?

Is stopping an actor sufficient to release memory? As I already mentioned, I do not keep any references to the actor besides the ones already existing within the actor system. Does the actor system remove them when I stop an actor?

I was trying to find such information in the documentation and Stackoverflow. I failed. To figure out what is going to happen, I created a test application.

How can I verify it?

What if I created a straightforward code that uses Akka to waste memory?

import akka.actor.{Actor, ActorSystem, Props}

import scala.util.Random
import scala.concurrent.duration._
import scala.concurrent.ExecutionContext.Implicits.global

case object Produce

object Application extends App {
  val actorSystem = ActorSystem()
  val actor = actorSystem.actorOf(Props[Parent])

  Console.println("Press enter")
  Console.readLine() //I need a few seconds to attach VisualVM to the process ;)

  actorSystem.scheduler.schedule(2 seconds, 200 millis, actor, Produce)
}

class Parent extends Actor {
  override def receive: Receive = {
    case Produce =>
      val child = context.system.actorOf(Props[Child])
      child ! Produce
  }
}

class Child extends Actor {
  val data = new Array[Byte](20000000)

  override def receive: Receive = {
    case Produce =>
      Random.nextBytes(data)
  }
}

When I looked at the memory usage chart in VisualVM I saw that:

Application crashing because of OutOfMemoryError
Application crashing because of OutOfMemoryError

And obviously the error I expected:

java.lang.OutOfMemoryError: Java heap space
 at Child.<init>(Application.scala:28)
 at sun.reflect.GeneratedConstructorAccessor2.newInstance(Unknown Source)
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
 at java.lang.Class.newInstance(Class.java:442)
 at akka.util.Reflect$.instantiate(Reflect.scala:44)
 at akka.actor.NoArgsReflectConstructor.produce(IndirectActorProducer.scala:105)
 at akka.actor.Props.newActor(Props.scala:213)
...

I limited the application memory to only 1GB. Note that the chart does not show 1GB of used memory. The application crashes when it tries to allocate memory, so the chart shows the last value before the crash.

Hypothesis

If the actor system removes references to an actor when the actor is stopped, garbage collector will remove the object from memory and the application will not crash. I should notice GC activity on VisualVM charts.

All I had to do, was sending a PoisonPill to the Child actor after generating the byte array:

class Child extends Actor {
  val data = new Array[Byte](20000000)

  override def receive: Receive = {
    case Produce =>
      Random.nextBytes(data)
      context.system.scheduler.scheduleOnce(1 second, self, PoisonPill)
  }
}

Now the program not only keeps running without any errors but also its memory usage chart looks as expected.

Garbage collector removes stopped actors from memory
Garbage collector removes stopped actors from memory

Conclusion

  • Do not forget about references stored internally by the actor system.

  • When you want to get rid of an Akka actor, simply stop it (if you do not keep any references to the actor instance).

  • Some of the actors you create may not be reusable. They won’t magically disappear, you have to remove them.

Older post

Scalar 2017

Scalar Conference 2017 — everything I liked

Newer post

4 reasons why TDD slows you down

It is easy to announce that TDD slows you down, but have you ever wondered why it happens? Is there anything you can do better?