Understanding in-loop filtering in the HEVC video standard
High Efficiency Video Coding (HEVC) is a video compression standard, a successor to H.264/MPEG-4 AVC (Advanced Video Coding), jointly developed by the ISO/IEC Moving Picture Experts Group (MPEG) and ITU-T Video Coding Experts Group (VCEG) as ISO/IEC 23008-2 MPEG-H Part 2 and ITU-T H.265. HEVC promises half bit-rate compared to current de-facto standard H.264 at a similar video quality and is expected to be deployed in a wide variety of video applications ranging from cell phones, broadcast, set-top box, video conferencing, video surveillance, automotive, etc.
The figure below shows the block diagram of the HEVC Video decoder with loop filtering, as shown with loop filtering highlighted. As shown, it is a cascading of two stages - namely de-blocking filtering (DBLK) and Sample adaptive offset (SAO) filtering to remove blocking artifacts causing during video encoding. The next two sections describe these two stages in detail.
2.0 DB-Blocking Filtering (DBLK)
This section explains de-blocking filtering as described in the HEVC Video standard . De-block filter operation can broadly be divided into two parts:
- Boundary Strength (aka BS) Computation on filter edge
- Actual Filter operation
Throughout the de-block filter operation the following convention has been used in the literature: Q pixels belong to the right side and P pixels belong to the left side of the “vertical” filter edge. Similarly, Q belongs to pixels below the horizontal edge and P above it for the “horizontal” filter edge.
2.1 Filter Edge Strength (aka BS)
BS [hor/ver][xpos][ypos] is computed on an 8x8 grid and takes values 0, 1 & 2 as shown in the figure below:
Although the filter is applied on every pixel edge on 8x8 grid (8x8 block contains 4 pixel edges), the property of boundary strength computation can be combined for all pixels for a 4-pixel segment. For strength computation, we assume every 4-pixel segment as one edge. Please note that Boundary strength computation depends on Current, Left and Top LCU LCUinfo.2.1.1 BS function Overview
BS is computed on an 8x8 grid, set to zero for picture and slice boundaries (if loop_filter_across_slice_enabled_flag = 0) and Tile boundaries (loop_filter_across_tile_enabled_flag = 0).
Only 8x8 pixel boundaries are filtered, which are Prediction Unit (PU) and/or Transform Unit (TU) boundaries, and get filtered as shown below.
BS derivation rules:
BS is set to 0 if an edge is neither a TU edge nor a PU edge of a CU or not 8x8 edge or picture boundary or slice boundary with across slice filtering off or tile boundary with across tile boundary filtering off
else if block p or q is intra-coded, set BS to 2
else if block p or q is in a TU containing non-zero coefficients, set BS to 1
else if block p and q has different reference pictures or different number of MVs, BS=1
else if block p and q have a single MV, and vertical or horizontal MV difference ≥ 4, BS=1
else if block p and q have 2 MVs, and at least one MV has vertical/hor. MV difference ≥ 4, BS=1
else BS = 0
Readers interested in more details can refer to HEVC Specification  for the following:
- Derivation of TU Boundaries for BS : Section 18.104.22.168
- Derivation of PU Boundaries for BS : Section 22.214.171.124
- Derivation of BS: Section 126.96.36.199