Behind the curtain: Rendering 16,000+ schematics in the cloud

CircuitLab, the browser-based analog simulator, upgraded its rendering engine. In order to deploy the new updates, however, it had to re-process the 18,000+  circuits that its community had saved since its start  in March. (Man, that’s a lotta circuits in less than 5 months.)

The CircuitLab folks calculated that rending each schematic took about 10 seconds of a single core of  CPU time per schematic, or about 44 core-hours.  But, as they write on their blog, “Who wants to wait 44 hours for results?”

The blog post is an interesting case study of running a problem that is perfectly suited for parallel processing. They write:

“In theory, if we had 16,000 CPU cores available to us, we could start each CPU working on rendering a separate circuit, and we’d be done all of them in just about 10 seconds. In practice, contention for shared resources like storage, network, and database make it impractical to scale linearly to this level, and the finite time and effort required to spin up/down new cores means that it simply doesn’t work that way. However, the fundamental idea of elastic cloud computing is that virtualized computing resources can be turned on and off as needed. Cloud computing shines where the workload is variable or unknown, the extreme being cases like ours where an infrequent but large computing job is need. Somewhere between 1 core and 16,000 cores lies a solution that gets our re-rendering job done much faster without requiring a major redesign of these potential bottlenecks.”

Click on over and read how they did it. It’s a case study in applying cloud computing.