rt-serializer-48

Usage

Garbage collection

"rt-serializer" maintains a priority queue of objects to serialize, or searches which may result in objects to serialize. When inserting into this queue, it does no checking if the object in question is already in the queue, or if the search will contain any results. These checks are done when the object reaches the front of the queue, or during periodic garbage collection.

During periodic garbage collection, the entire queue is swept for objects which have already been serialized, occur more than once in the queue, and searches which contain no results in the database. This is done to reduce the memory footprint of the serialization process, and is triggered when enough new objects have been placed in the queue. This parameter is tunable via the "--gc" parameter, which defaults to running garbage collection every 5,000 objects inserted into the queue; smaller numbers will result in more frequent garbage collection.

The default of 5,000 is roughly tuned based on a database with several thousand tickets, but optimal values will vary wildly depending on database configuration and size. Values as low as 25 have provided speedups with smaller databases; if speed is a factor, experimenting with different "--gc" values may be helpful. Note that there are significant boundary condition changes in serialization rate, as the queue empties and fills, causing the time estimates to be rather imprecise near the start and end of the process.

Setting "--gc" to 0 turns off all garbage collection. Be aware that this will bloat the memory usage of the serializer. Any negative value for "--gc" turns off periodic garbage collection and instead objects already serialized or in the queue are checked for at the time they would be inserted.