Getting real with the real-time clock
Tales From The Cube: The engineering team responsible for real-time motion control of 30 giant antennas composing the world's largest radio telescope must determine why antennas are drifting off target.
By Vishwas Vaidya, Tata Motors -- EDN, February 5, 2009
In the 1990s, I was the lead servo-system engineer for the world's largest radio telescope (Reference 1). Our team was responsible for real-time motion control of 30 45m-diameter giant antennas that composed the radio telescope. Astronomers sitting at a central console focused the antenna position on the target—the radio star under observation. Because the rotation of the earth causes apparent drift in the position of the target, the antenna had to track the target by moving accordingly. The servo system was supposed to “servo track” this motion under the supervisory control of the central computer. By 1993, a few of the prototype antenna systems were undergoing proving trials, which uncovered many painful surprises. The chief astronomer reported that the antenna was drifting off the target by minutes—and that was just one of our problems.
Surprisingly, the antenna time would remain accurate when antennas were not moving. To catch the culprit, we installed a spy counter in an embedded timekeeping routine. The counter would drift only when antennas were in tracking motion. The system would then periodically send the counter value to the central computer, which would record the signature of all the events occurring when the counter reported a miss. Heavy traffic comprising instantaneous trajectory information always coincided with the time slippage.
|
Analysis proved our worst doubts true. The onboard software RTC (real-time clock) was lagging behind whenever the CPU was busy servicing the dense traffic arriving from the central computer. Because this traffic was absent when the antenna was still, the software RTC in that case would be accurate. This board unfortunately lacked a hardware-RTC chip; hence, we had to implement a software RTC, which proved accurate enough during bench trials. Apparently, we could not accurately simulate the dense traffic from the central computer.
The project received its finances from the public, and redesign would have been suicidal. We had two options: Provide the single-board computer with a hardware RTC without touching the PCB (printed-circuit board) or wind up the project.
Fortunately, PCs were then widely using the cheapest RTC with nonvolatile RAM that we could afford. Unfortunately, the chip included the notorious multiplexed address/data bus. A little thought revealed that a bit of smart coding can indeed infuse enough intelligence inside those parallel I/Os to mimic a multiplexed bus. We successfully tested the idea and deployed it after necessary software changes (Reference 2).
Once we equipped the antennas with the RTC daughterboards, the antennas started obediently counting the time with accuracy within milliseconds. Modern embedded-system engineers may not make the mistakes we made during the design stage. However, this story amply illustrates how a clever piece of engineering can always pull you out of a crisis!
References
-
“Giant Metrewave Radio Telescope,” Tata Institute of Fundamental Research.
-
Vaidya, Vishwas, “I/O bus emulates multiplexed address/data bus,” EDN, Feb 15, 1996, pg 120.
-
Dear Anonymous,
Milisecond accuracy was not required for our application! Also please note that the story dates back to early nineties! Very fact that EDN accepted this as a design idea that time proves the elegence of the solution
Regards
Author
Vishwas Vaidya - 2009-11-7 04:21:00 PDT -
I faced a similar problem with a single board controller that already gone through manufacturing and we ended up using a Dallas Semiconductor "Smart Watch " socket in one of the static ram sockets on the board which added the rtc and battery back up for the clock.
Bruce Westberg - 2009-18-2 14:01:00 PST -
Millisecond synchronization is sooooo out of date. Coming soon to an Ethernet port near you: synchronization in the submicrosecond range using IEEE1588-2008.
Anonymous - 2009-18-2 12:55:00 PST
