Minimizing Video Processing Design Time with FPGA Development Kits and Reference Designs

Introduction
Video standards and methods of encoding/decoding have made tremendous progress over the last decade with the availability of large system-on-a-chip (SOC) solutions using ASSPs, ASICs, or FPGAs. The majority of broadcast content is already streaming in high definition (HD) and is moving from an interlace format to a progressive format. Today’s broadcast equipment must be capable of handling all types of video formats and resolutions. Video processing with FPGAs still holds a key advantage over other types of technologies due to the inherent flexibility and scalability of these devices.

The 1080p 60/120Hz Video
Video displays have evolved from bulky CRTs to the popular LCDs for today’s professional video monitor. For high end color graphics, CRTs offer some advantages because LCDs can only display the colors available in the pixels, and so they have less color depth than CRTs. LCD weaknesses like motion blur and film judder affect the video monitoring experience drastically. Existing LCD monitors use motion-adaptive deinterlacing techniques to create the progressive frame. The conversion of 24p film sources to the LCD refresh rate of 50 Hz or 60 Hz causes film judder. This is because the 24 frames are repeated two or three times (2:2 or 3:2 pull-down) to match the LCD panel’s refresh rate. One method is to remove any annoying film judder by up converting 1080p at 60/120 Hz using motion-compensating deinterlacing by inserting additional interpolated pictures into the original picture sequence. This virtually removes motion blur on high end 60/120 Hz/1080p LCD monitors. Many standard consumer types of ASSPs are available to address this issue, but for higher end broadcast quality video monitors, more video processing power and flexibility is needed to enhance the performance of the video output quality.

The FPGA Alternative for the Studio Switcher Example
FPGA SOCs provide an alternative to the traditional ASIC or ASSP for video processing. Designers can select either a low-cost or a high-end FPGA to fulfill the system requirements. Figure 1 shows a broadcast video switcher, the gateway of the transmission studio, that processes streams of uncompressed video from standard definition (SD) to HD, including 1080p.

The uncompressed SDI video streams are routed into a video special effect block that adds local text information, up/down cross resolution conversion, or video overlay. The format converter provides up and down conversion between HD (1080i/p and 720p) and SD (480i and 576i). Before the edited streams are routed out for transmission, they are also routed to a local multi-viewer display for content monitoring. The video processing blocks are the essential building blocks of the video switcher. FPGAs are ideal for this type of video system design with their internal parallel digital signal processing (DSP) blocks, an abundance of embedded memory blocks, a large number of registers, integrated transceivers that can handle 1080p 3G-SDI, and high-speed memory interfaces. A typical switcher can support multiple SDI ports. Each output stream represents an edited program for live broadcast or storing for later retrieval.

Figure 1. Broadcast Switcher Block Diagram

Video Interfaces
SDI is the industry standard for broadcasting uncompressed video delivery within the studio. High definition multimedia interface (HDMI) and DVI are also used for local display monitors. The data rate for each interface is listed in Table 1. The video resolution can go up to a maximum of 1920×1200p60 depending on the application.

Table 1. Video Interface Data Rates

Video Processor Building Blocks and Design Tools
The most common video building blocks are already available and can be easily incorporated into a new design. Altera’s Video and Image Processing (VIP) Suite are examples of a ready-to-use broadcast resolution video building blocks. The blocks include scalar, deinterlacer, frame buffer, color plane sequencer, clipper, color space converter, gamma corrector, chroma resampler, alpha blending mixer, 2D FIR filter, 2D Median filter, line buffer compiler, and even a test pattern generator for debug purposes.

A reference design framework tool is also part of the video library. This enables designers to use a video design framework that promotes the fastest design cycle for video applications. System design tools such as SOPC Builder allows an automated way of generating control and arbitration logic in a time-efficient manner. This reference design is used as a starting point for a final video datapath design. For a multi-viewer application, Figure 2 shows a two channel video input processing that utilizes the majority of the building block. The two SDI video sources are clocked in through the clocked video input (CVI) blocks, followed by clipping, and a color space converter with deinterlacing and scaling. The video source can also be format converted in the scaling block and final mixed onto one video screen through the DVI output port via the clocked video output (CVO).

Figure 2. Multi-viewer Application with VIP Building Blocks

Development Kits
A wide variety of low-cost and high-end FPGA video development kits and matching video daughter cards are available to designers. These video development kits are equipped with the common video interfaces. Vendors offer a wide range of development kits that contain everything an engineer needs to create and implement a design in only a few hours.

Reference Designs
Reference design plays an important role as a key time saving factor. They are often used to kick-start user projects by providing a low level template that includes the project tool, environmental settings, and input/output pin assignments with required timing constraints.

Two example reference designs are: 1) a simple SDRAM loop-through that buffers an incoming 1080p video signal through the top and bottom banks of DDR2 found on the Bitec Cyclone III Video Development Kit and 2) a more sophisticated example of pipelined video processing in which a NTSC video signal is overlaid onto a 1080p full HD video signal.

SDRAM Loop-Through
Image buffering is used to synchronize video sources with the local processor clock and frame rate. For processing systems that operate at the same frame rate, double buffering can be used. When the video processing system alters the incoming frame-rate by either up- or down-sampling triple buffering is necessary.

This reference design demonstrates the implementation of two cascaded triple frame buffers (see Figure 3). Frame buffering is used in many video processing systems.

Figure 3. 1080p SDRAM Loop-Through Block Diagram

A raw DVI signal in RGB444 format is input to the DVI port on this video development kit and converted into a standard Avalon Imagestream. The Image Stream (IS) is an Altera Video Processing Suit representation of the video signal. The IS then is triple frame buffered in an external DDR2 SDRAM. The image is then passed to a second triple frame buffer before being transmitted to the video output. The output video is driven at a local pixel clock which does not have to be the same as the input clock frequency as a result of the triple frame buffering.

The triple frame buffer design is built without the need for RTL coding as all the processing elements are standard SoPC components found in the VIP Suite. The Quartus II SoPC Editor diagram for the design is shown in Figure 4. Available components are selected from the component browser in the right-hand pane.

Figure 4. 1080p SDRAM Loop-Through SoPC Editor Diagram

The "Connections" column represents the interconnectivity through the Avalon fabric. Each component in the "Module Name" column is characterized by its input and output interfaces. These interfaces are either Avalon Streaming or Avalon Master and Slave memory mapped as shown in the Description column. The "Clock" column allows the designer to specify a clock from the “Clock Settings” pane to drive the component and/or Avalon interface. Finally, an editable memory mapped address range indicates the memory mapped address space a slave interface appears to its driving master.

The Avalon IS video stream originates from the CVI component from which it is fed to a “Frame Buffer” The Frame Buffer has both an Avalon stream input/output through which the buffered video stream is passed and a read and write Avalon master interface which connects to the DDR2 controller components.    

The individual SoPC components are parameterized using a component editor dialog. The CVI editor dialog is shown in Figure 5.
 

Figure 5. Clocked Video Input Parameterization Dialogue

After system generation a SoPC component symbol is available for inclusion in the main design. The SDRAM loop-through design is shown below. In this case, the design also includes an FPGA PLL device to generate the 156 MHz pixel clock for the video output stream. The FPGA is connected to the outside world using the I/O pins as shown.  

Figure 6. Top Level Design Schematic

Users can customize the design by introducing other standard video processing blocks or implementing RTL processing elements to integrate into the video pipeline. The following example demonstrates an elaboration of the simple SDRAM loop-through to perform a full video processing pipeline.

1080p Picture-In-Picture
Found primarily on more expensive televisions, picture-in-picture (PiP) allows watching a second video stream in a small, sub-window. More elaborate versions can resize the window, move it around the screen, create still or multiple still images, or simply divide the screen into two same-size pictures, often called ‘picture-outside-picture’ (POP).

This reference design demonstrates the use of this video development kit for PiP on 1080p full HD video signals. An NTSC video signal is overlaid onto a 1080p video stream to generate a 1080P, PiP output video stream. This reference design was built from the simple SDRAM loop-through discussed previously. A block diagram of the design is shown in Figure 7.

Figure 7. 1080P PiP Reference Design

A 1080p RGB444 video signal is provided by a DVI interface on the video development kit and triple frame buffered to allow synchronization to the local pixel clock and frame rate. The buffered image stream is then fed into a mixer component.

The YUV422 NTSC signal is input to one of four channels available on this video development kit. This signal is first clipped, re-sampled into YUV444 and then converted into RGB444 via a color space converter. The video stream is then de-interlaced and scaled to 800 x 600 before being synchronized to the local output pixel and frame rate by a triple frame buffer. The synchronized image stream is then presented to the mixer component.

The mixer component has a control port that allows a Nios II embedded processor to activate and adjust the position of the overlay in real-time. The embedded processor is also connected to a FPGA ram block for code execution. An I2C peripheral is also included for configuring the various onboard devices.

The SoPC Editor implementation of the 1080p PiP design is shown in Figure 8. The composite video signal path is shown in red and the 1080p signal path shown in green with the combined PiP video steam shown in blue. The embedded processor runs at a slower 50 MHz clock and requires a “Clock Crossing Bridge” in order to interface to the main video pipeline which runs at the DDR2 166 MHz clock rate.

As with the SDRAM loop-through design discussed earlier, there is no RTL code used within this design.

Figure 8. 1080P PiP SoPC Editor

A more complex version of the PiP design is available that combines the four available NTSC inputs and overlays them onto the 1080p background video. The design extends the single PiP example discussed here by including 4-copies of the NTSC video pipeline.

Conclusion
Professional video systems demand high performance processing combined with a wide range of application requirements from interface types to resolution sizes. Having a dedicated ASIC or ASSP make it difficult to satisfy the on-going new broadcast technology rollouts. FPGAs have become the implementation platform of choice. FPGA and third party vendors have responded by providing a suite of IP, design tools, reference designs, and development kits that help rapid design, development, and prototyping of such systems for quick time to market.

About the Authors

Tam Do, Senior Technical Marketing Manager, Broadcast/Automotive/ Consumer Business Unit.  As senior technical marketing manager of the Broadcast/Consumer Applications Business Unit, Tam Do is responsible for all technical and marketing issues related to the digital broadcast, automotive, and consumer electronics industries. Mr. Do joined Altera in June 2003. Before that time, he was most recently the application design manager of LSI Logic’s Consumer Product Group, where he focused on the development design of an ASSP evaluation system and software for the set-top box industry. Mr. Do holds a BSEE from the University of Nevada Reno, and has close to 20 years of electronics system experience with LSI Logic, Stratex Network, and Verizon Corporation.

Andy Robertson is founder and Director of Bitec. With over 20 years experience in the electronics and computing industry, he has a wide range of experience across various market segments including automotive, rail, and aerospace. Mr Robertson graduated with First Class Honours from Southampton University in the UK and holds both a Masters from UMIST, UK and a PhD from La Universidad Pontificia Comillas, Madrid, Spain.