Video surveillance has been making news in the San Francisco area, in a negative way. A study found that installation of video cameras all over an at-risk housing project had not contributed to the solution of one crime, let alone prevention. A report on an area airport stated that there was no staff to watch the monitors for the numerous video cameras scattered around critical security points. Another report questioned whether the city of San Francisco was getting any value at all from the numerous expensive street cameras it had installed, for similar reasons.

All of this underlines a lesson we didn’t learn from the rollout of three generations of cellular service. It doesn’t matter how well-designed the hardware is if you let uninformed and unmotivated people integrate and use the end-system. You can turn the best collection of high-resolution color surveillance cameras into trash by simply not watching the images.

But this is not intended as social commentary—rather, there is an architectural lesson here. Systems, and SoCs, have to be designed with the limitations of the end-user in mind. And in applications outside the electronics industry, those limitations are often severe. For instance, if you know that end-user institutions are not going to budget for people to watch monitors, you have to provide for video analytics in your hardware architecture.

One illustration of this point is in a recent reference design provided by configurable video processor vendor Stretch Inc. The design incorporates Stretch’s recently-announced S6 SoC, which in turn includes a Tensilica Xtensa RISC core, a large configurable arithmetic coprocessor and more specialized programmable accelerators for encryption, motion and entropy coding. Thus except for the configurable fabric, the hardware is not that different from a number of other announcements in the surveillance space.

Unfortunately, the software needs to be close to the cameras, since latency between identification of an object and commands to the cameras is critical, and since one of the important roles of the software is to reduce the bandwidth required—and the quality of service demanded, in video-over-IP scenarios—between the camera clusters and the central facility.

The problem is, video analytics algorithms are proprietary, and not blessed by even the minimal standardization granted to video compression algorithms. This is entirely appropriate to a field whose algorithms are still research areas, but it means that there is virtually no chance of providing optimized hardware to support the algorithms yet. Chip architects have to make powerful general-purpose array processing as accessible as possible to these software developers.

And that of course brings us back to architecture. If architects think of video surveillance as purely an application for variable-bit-rate, high-quality compression, the obvious solution is to move as far as possible toward fixed-function hardware. That will reduce chip size and power for a given number of channels. But if a big part of the job will be executing video analytics, the architect should shift toward general-purpose image-processing resources and simple programming models—just the opposite direction.

The hard part is that no one is going to specify in their system requirements document that the end-user organization will provision their IP network inadequately and fail to hire enough security personnel. Those facts—very much to the point for the chip architect—have to be learned in the field. The connection between chip architects and system vendors is constantly growing more intimate.