1 <title>Video Overlay Interface</title> 2 <subtitle>Also known as Framebuffer Overlay or Previewing</subtitle> 3 4 <para>Video overlay devices have the ability to genlock (TV-)video 5into the (VGA-)video signal of a graphics card, or to store captured 6images directly in video memory of a graphics card, typically with 7clipping. This can be considerable more efficient than capturing 8images and displaying them by other means. In the old days when only 9nuclear power plants needed cooling towers this used to be the only 10way to put live video into a window.</para> 11 12 <para>Video overlay devices are accessed through the same character 13special files as <link linkend="capture">video capture</link> devices. 14Note the default function of a <filename>/dev/video</filename> device 15is video capturing. The overlay function is only available after 16calling the &VIDIOC-S-FMT; ioctl.</para> 17 18 <para>The driver may support simultaneous overlay and capturing 19using the read/write and streaming I/O methods. If so, operation at 20the nominal frame rate of the video standard is not guaranteed. Frames 21may be directed away from overlay to capture, or one field may be used 22for overlay and the other for capture if the capture parameters permit 23this.</para> 24 25 <para>Applications should use different file descriptors for 26capturing and overlay. This must be supported by all drivers capable 27of simultaneous capturing and overlay. Optionally these drivers may 28also permit capturing and overlay with a single file descriptor for 29compatibility with V4L and earlier versions of V4L2.<footnote> 30 <para>A common application of two file descriptors is the 31XFree86 <link linkend="xvideo">Xv/V4L</link> interface driver and 32a V4L2 application. While the X server controls video overlay, the 33application can take advantage of memory mapping and DMA.</para> 34 <para>In the opinion of the designers of this API, no driver 35writer taking the efforts to support simultaneous capturing and 36overlay will restrict this ability by requiring a single file 37descriptor, as in V4L and earlier versions of V4L2. Making this 38optional means applications depending on two file descriptors need 39backup routines to be compatible with all drivers, which is 40considerable more work than using two fds in applications which do 41not. Also two fd's fit the general concept of one file descriptor for 42each logical stream. Hence as a complexity trade-off drivers 43<emphasis>must</emphasis> support two file descriptors and 44<emphasis>may</emphasis> support single fd operation.</para> 45 </footnote></para> 46 47 <section> 48 <title>Querying Capabilities</title> 49 50 <para>Devices supporting the video overlay interface set the 51<constant>V4L2_CAP_VIDEO_OVERLAY</constant> flag in the 52<structfield>capabilities</structfield> field of &v4l2-capability; 53returned by the &VIDIOC-QUERYCAP; ioctl. The overlay I/O method specified 54below must be supported. Tuners and audio inputs are optional.</para> 55 </section> 56 57 <section> 58 <title>Supplemental Functions</title> 59 60 <para>Video overlay devices shall support <link 61linkend="audio">audio input</link>, <link 62linkend="tuner">tuner</link>, <link linkend="control">controls</link>, 63<link linkend="crop">cropping and scaling</link> and <link 64linkend="streaming-par">streaming parameter</link> ioctls as needed. 65The <link linkend="video">video input</link> and <link 66linkend="standard">video standard</link> ioctls must be supported by 67all video overlay devices.</para> 68 </section> 69 70 <section> 71 <title>Setup</title> 72 73 <para>Before overlay can commence applications must program the 74driver with frame buffer parameters, namely the address and size of 75the frame buffer and the image format, for example RGB 5:6:5. The 76&VIDIOC-G-FBUF; and &VIDIOC-S-FBUF; ioctls are available to get 77and set these parameters, respectively. The 78<constant>VIDIOC_S_FBUF</constant> ioctl is privileged because it 79allows to set up DMA into physical memory, bypassing the memory 80protection mechanisms of the kernel. Only the superuser can change the 81frame buffer address and size. Users are not supposed to run TV 82applications as root or with SUID bit set. A small helper application 83with suitable privileges should query the graphics system and program 84the V4L2 driver at the appropriate time.</para> 85 86 <para>Some devices add the video overlay to the output signal 87of the graphics card. In this case the frame buffer is not modified by 88the video device, and the frame buffer address and pixel format are 89not needed by the driver. The <constant>VIDIOC_S_FBUF</constant> ioctl 90is not privileged. An application can check for this type of device by 91calling the <constant>VIDIOC_G_FBUF</constant> ioctl.</para> 92 93 <para>A driver may support any (or none) of five clipping/blending 94methods:<orderedlist> 95 <listitem> 96 <para>Chroma-keying displays the overlaid image only where 97pixels in the primary graphics surface assume a certain color.</para> 98 </listitem> 99 <listitem> 100 <para>A bitmap can be specified where each bit corresponds 101to a pixel in the overlaid image. When the bit is set, the 102corresponding video pixel is displayed, otherwise a pixel of the 103graphics surface.</para> 104 </listitem> 105 <listitem> 106 <para>A list of clipping rectangles can be specified. In 107these regions <emphasis>no</emphasis> video is displayed, so the 108graphics surface can be seen here.</para> 109 </listitem> 110 <listitem> 111 <para>The framebuffer has an alpha channel that can be used 112to clip or blend the framebuffer with the video.</para> 113 </listitem> 114 <listitem> 115 <para>A global alpha value can be specified to blend the 116framebuffer contents with video images.</para> 117 </listitem> 118 </orderedlist></para> 119 120 <para>When simultaneous capturing and overlay is supported and 121the hardware prohibits different image and frame buffer formats, the 122format requested first takes precedence. The attempt to capture 123(&VIDIOC-S-FMT;) or overlay (&VIDIOC-S-FBUF;) may fail with an 124&EBUSY; or return accordingly modified parameters..</para> 125 </section> 126 127 <section> 128 <title>Overlay Window</title> 129 130 <para>The overlaid image is determined by cropping and overlay 131window parameters. The former select an area of the video picture to 132capture, the latter how images are overlaid and clipped. Cropping 133initialization at minimum requires to reset the parameters to 134defaults. An example is given in <xref linkend="crop" />.</para> 135 136 <para>The overlay window is described by a &v4l2-window;. It 137defines the size of the image, its position over the graphics surface 138and the clipping to be applied. To get the current parameters 139applications set the <structfield>type</structfield> field of a 140&v4l2-format; to <constant>V4L2_BUF_TYPE_VIDEO_OVERLAY</constant> and 141call the &VIDIOC-G-FMT; ioctl. The driver fills the 142<structname>v4l2_window</structname> substructure named 143<structfield>win</structfield>. It is not possible to retrieve a 144previously programmed clipping list or bitmap.</para> 145 146 <para>To program the overlay window applications set the 147<structfield>type</structfield> field of a &v4l2-format; to 148<constant>V4L2_BUF_TYPE_VIDEO_OVERLAY</constant>, initialize the 149<structfield>win</structfield> substructure and call the 150&VIDIOC-S-FMT; ioctl. The driver adjusts the parameters against 151hardware limits and returns the actual parameters as 152<constant>VIDIOC_G_FMT</constant> does. Like 153<constant>VIDIOC_S_FMT</constant>, the &VIDIOC-TRY-FMT; ioctl can be 154used to learn about driver capabilities without actually changing 155driver state. Unlike <constant>VIDIOC_S_FMT</constant> this also works 156after the overlay has been enabled.</para> 157 158 <para>The scaling factor of the overlaid image is implied by the 159width and height given in &v4l2-window; and the size of the cropping 160rectangle. For more information see <xref linkend="crop" />.</para> 161 162 <para>When simultaneous capturing and overlay is supported and 163the hardware prohibits different image and window sizes, the size 164requested first takes precedence. The attempt to capture or overlay as 165well (&VIDIOC-S-FMT;) may fail with an &EBUSY; or return accordingly 166modified parameters.</para> 167 168 <table pgwide="1" frame="none" id="v4l2-window"> 169 <title>struct <structname>v4l2_window</structname></title> 170 <tgroup cols="3"> 171 &cs-str; 172 <tbody valign="top"> 173 <row> 174 <entry>&v4l2-rect;</entry> 175 <entry><structfield>w</structfield></entry> 176 <entry>Size and position of the window relative to the 177top, left corner of the frame buffer defined with &VIDIOC-S-FBUF;. The 178window can extend the frame buffer width and height, the 179<structfield>x</structfield> and <structfield>y</structfield> 180coordinates can be negative, and it can lie completely outside the 181frame buffer. The driver clips the window accordingly, or if that is 182not possible, modifies its size and/or position.</entry> 183 </row> 184 <row> 185 <entry>&v4l2-field;</entry> 186 <entry><structfield>field</structfield></entry> 187 <entry>Applications set this field to determine which 188video field shall be overlaid, typically one of 189<constant>V4L2_FIELD_ANY</constant> (0), 190<constant>V4L2_FIELD_TOP</constant>, 191<constant>V4L2_FIELD_BOTTOM</constant> or 192<constant>V4L2_FIELD_INTERLACED</constant>. Drivers may have to choose 193a different field order and return the actual setting here.</entry> 194 </row> 195 <row> 196 <entry>__u32</entry> 197 <entry><structfield>chromakey</structfield></entry> 198 <entry>When chroma-keying has been negotiated with 199&VIDIOC-S-FBUF; applications set this field to the desired pixel value 200for the chroma key. The format is the same as the pixel format of the 201framebuffer (&v4l2-framebuffer; 202<structfield>fmt.pixelformat</structfield> field), with bytes in host 203order. E. g. for <link 204linkend="V4L2-PIX-FMT-BGR32"><constant>V4L2_PIX_FMT_BGR24</constant></link> 205the value should be 0xRRGGBB on a little endian, 0xBBGGRR on a big 206endian host.</entry> 207 </row> 208 <row> 209 <entry>&v4l2-clip; *</entry> 210 <entry><structfield>clips</structfield></entry> 211 <entry>When chroma-keying has <emphasis>not</emphasis> 212been negotiated and &VIDIOC-G-FBUF; indicated this capability, 213applications can set this field to point to an array of 214clipping rectangles.</entry> 215 </row> 216 <row> 217 <entry></entry> 218 <entry></entry> 219 <entry>Like the window coordinates 220<structfield>w</structfield>, clipping rectangles are defined relative 221to the top, left corner of the frame buffer. However clipping 222rectangles must not extend the frame buffer width and height, and they 223must not overlap. If possible applications should merge adjacent 224rectangles. Whether this must create x-y or y-x bands, or the order of 225rectangles, is not defined. When clip lists are not supported the 226driver ignores this field. Its contents after calling &VIDIOC-S-FMT; 227are undefined.</entry> 228 </row> 229 <row> 230 <entry>__u32</entry> 231 <entry><structfield>clipcount</structfield></entry> 232 <entry>When the application set the 233<structfield>clips</structfield> field, this field must contain the 234number of clipping rectangles in the list. When clip lists are not 235supported the driver ignores this field, its contents after calling 236<constant>VIDIOC_S_FMT</constant> are undefined. When clip lists are 237supported but no clipping is desired this field must be set to 238zero.</entry> 239 </row> 240 <row> 241 <entry>void *</entry> 242 <entry><structfield>bitmap</structfield></entry> 243 <entry>When chroma-keying has 244<emphasis>not</emphasis> been negotiated and &VIDIOC-G-FBUF; indicated 245this capability, applications can set this field to point to a 246clipping bit mask.</entry> 247 </row> 248 <row> 249 <entry spanname="hspan"><para>It must be of the same size 250as the window, <structfield>w.width</structfield> and 251<structfield>w.height</structfield>. Each bit corresponds to a pixel 252in the overlaid image, which is displayed only when the bit is 253<emphasis>set</emphasis>. Pixel coordinates translate to bits like: 254<programlisting> 255((__u8 *) <structfield>bitmap</structfield>)[<structfield>w.width</structfield> * y + x / 8] & (1 << (x & 7))</programlisting></para><para>where <structfield>0</structfield> ≤ x < 256<structfield>w.width</structfield> and <structfield>0</structfield> ≤ 257y <<structfield>w.height</structfield>.<footnote> 258 <para>Should we require 259 <structfield>w.width</structfield> to be a multiple of 260 eight?</para> 261 </footnote></para><para>When a clipping 262bit mask is not supported the driver ignores this field, its contents 263after calling &VIDIOC-S-FMT; are undefined. When a bit mask is supported 264but no clipping is desired this field must be set to 265<constant>NULL</constant>.</para><para>Applications need not create a 266clip list or bit mask. When they pass both, or despite negotiating 267chroma-keying, the results are undefined. Regardless of the chosen 268method, the clipping abilities of the hardware may be limited in 269quantity or quality. The results when these limits are exceeded are 270undefined.<footnote> 271 <para>When the image is written into frame buffer 272memory it will be undesirable if the driver clips out less pixels 273than expected, because the application and graphics system are not 274aware these regions need to be refreshed. The driver should clip out 275more pixels or not write the image at all.</para> 276 </footnote></para></entry> 277 </row> 278 <row> 279 <entry>__u8</entry> 280 <entry><structfield>global_alpha</structfield></entry> 281 <entry>The global alpha value used to blend the 282framebuffer with video images, if global alpha blending has been 283negotiated (<constant>V4L2_FBUF_FLAG_GLOBAL_ALPHA</constant>, see 284&VIDIOC-S-FBUF;, <xref linkend="framebuffer-flags" />).</entry> 285 </row> 286 <row> 287 <entry></entry> 288 <entry></entry> 289 <entry>Note this field was added in Linux 2.6.23, extending the structure. However 290the <link linkend="vidioc-g-fmt">VIDIOC_G/S/TRY_FMT</link> ioctls, 291which take a pointer to a <link 292linkend="v4l2-format">v4l2_format</link> parent structure with padding 293bytes at the end, are not affected.</entry> 294 </row> 295 </tbody> 296 </tgroup> 297 </table> 298 299 <table pgwide="1" frame="none" id="v4l2-clip"> 300 <title>struct <structname>v4l2_clip</structname><footnote> 301 <para>The X Window system defines "regions" which are 302vectors of struct BoxRec { short x1, y1, x2, y2; } with width = x2 - 303x1 and height = y2 - y1, so one cannot pass X11 clip lists 304directly.</para> 305 </footnote></title> 306 <tgroup cols="3"> 307 &cs-str; 308 <tbody valign="top"> 309 <row> 310 <entry>&v4l2-rect;</entry> 311 <entry><structfield>c</structfield></entry> 312 <entry>Coordinates of the clipping rectangle, relative to 313the top, left corner of the frame buffer. Only window pixels 314<emphasis>outside</emphasis> all clipping rectangles are 315displayed.</entry> 316 </row> 317 <row> 318 <entry>&v4l2-clip; *</entry> 319 <entry><structfield>next</structfield></entry> 320 <entry>Pointer to the next clipping rectangle, NULL when 321this is the last rectangle. Drivers ignore this field, it cannot be 322used to pass a linked list of clipping rectangles.</entry> 323 </row> 324 </tbody> 325 </tgroup> 326 </table> 327 328 <!-- NB for easier reading this table is duplicated 329 in the vidioc-cropcap chapter.--> 330 331 <table pgwide="1" frame="none" id="v4l2-rect"> 332 <title>struct <structname>v4l2_rect</structname></title> 333 <tgroup cols="3"> 334 &cs-str; 335 <tbody valign="top"> 336 <row> 337 <entry>__s32</entry> 338 <entry><structfield>left</structfield></entry> 339 <entry>Horizontal offset of the top, left corner of the 340rectangle, in pixels.</entry> 341 </row> 342 <row> 343 <entry>__s32</entry> 344 <entry><structfield>top</structfield></entry> 345 <entry>Vertical offset of the top, left corner of the 346rectangle, in pixels. Offsets increase to the right and down.</entry> 347 </row> 348 <row> 349 <entry>__u32</entry> 350 <entry><structfield>width</structfield></entry> 351 <entry>Width of the rectangle, in pixels.</entry> 352 </row> 353 <row> 354 <entry>__u32</entry> 355 <entry><structfield>height</structfield></entry> 356 <entry>Height of the rectangle, in pixels.</entry> 357 </row> 358 </tbody> 359 </tgroup> 360 </table> 361 </section> 362 363 <section> 364 <title>Enabling Overlay</title> 365 366 <para>To start or stop the frame buffer overlay applications call 367the &VIDIOC-OVERLAY; ioctl.</para> 368 </section> 369