Video Acceleration API

Overview

The main motivation for VA-API is to enable hardware-accelerated video decode at various entry-points (VLD, IDCT, motion compensation, deblocking) for the prevailing coding standards today (MPEG-2, MPEG-4 ASP/H.263, MPEG-4 AVC/H.264, H.265/HEVC, and VC-1/WMV3). Extending XvMC was considered, but due to its original design for MPEG-2 MotionComp only, it made more sense to design an interface from scratch that can fully expose the video decode capabilities in today's GPUs.

Supported hardware and drivers

As of 2022, VA-API is natively supported by:

• Intel Quick Sync open-source drivers for Linux
• Mesa open-source drivers for AMD and Nvidia graphics cards
• AMDGPU-PRO drivers for AMD graphics cards on Linux
• libva-vdpau-driver for cards supported by VDPAU
• Direct3D 12 implementations with the VAOn12 driver

Supported video codecs

VA-API currently supports these video codecs in the official mainline version, but note that exactly which video codecs are supported depends on the hardware and the driver's capabilities.

• MPEG-2 decode acceleration Main Profile
• VC-1 / WMV3 decode acceleration Advanced Profile
• MPEG-4 Part 2 (H.263) (a.k.a. MPEG-4 SP / MPEG-4 ASP, more commonly known as Xvid) decode acceleration
• H.264 AVC encode acceleration Main Profile
• H.264 AVC decode acceleration High Profile
  • H.264 / AVC Hardware Variable Length Decoding (VLD) - CABAC
  • H.264 / AVC Hardware Variable Length Decoding (VLD) - CAVLC
  • H.264 / AVC Hardware Inverse Transform (IT)
  • H.264 / AVC Hardware Motion Compensation (HWMC)
  • H.264 / AVC Hardware In-Loop Deblocking (ILDB)
• H.265/HEVC encode acceleration
• H.265/HEVC decode acceleration
• VP9 8-bit encode acceleration
• VP9 8-bit and 10-bit decode acceleration
• AV1 8-bit and 10-bit encode acceleration
• AV1 8-bit and 10-bit decode acceleration

Processes that can be accelerated with VA-API

Video decoding and post-processing processes that can be offloaded and accelerated if both the device drivers and GPU hardware supports them:

• Motion compensation
• Inverse discrete cosine transform
• In-loop deblocking filter
• Intra-frame prediction
• variable-length decoding, more commonly known as slice-level acceleration
• Bitstream processing (Context-adaptive variable-length coding/Context-adaptive binary arithmetic coding)

Software architecture

The current interface is window system independent, so that it can potentially be used with graphics sub-systems other than the DRI (Direct Rendering Infrastructure) in X Window System, such as direct with framebuffer, and it can work with third-party DRM (Direct Rendering Manager) libraries. In a nutshell, it is a scheme to pass various types of data buffers from the application to the GPU for decoding or encoding a compressed bit-stream.

Software supporting VA-API

• Chromium
• Bluecherry DVR client (starting from release 2.2.6)
• datarhei Restreamer real-time live video streaming
• Emby media server (starting from release 3.0.6400)
• ffmpeg, a command line tool from the FFmpeg project (starting from 3.1)
• Firefox (On Wayland since release 78 and on X11 since release 80)
• Fluendo
• Frigate NVR
• Gnash Flash / SWF player
• GStreamer through
• Helix media player (Linux)
• Jellyfin media server
• Kodi (formerly XBMC Media Center) (Linux)
• Kodibuntu (formerly XBMC Live) (Linux Live CD/USB operating-system)
• libavcodec
• MPlayer with custom patches and its fork mpv (native)
• MythTV (starting from release 0.25)
• OBS Studio
• OpenShot
• VLC media player (starting from release 1.1.0)
• Xine (via "xine-lib-vaapi" library) since Version 1.2.3

References

References

1. "VA-API Video Acceleration On Intel Medfield - Phoronix".
2. "Video4Linux2: Path to a Standardized Video Codec API".
3. Nathan Willis. (2009-07-01). "VA API slowly, but surely, making progress".
4. "Mplayer, FFmpeg Gain VA-API Support - Phoronix".
5. "vaapi".
6. "Hardware video acceleration". wiki.ArchLinux.org.
7. Vilerino, Sil. (2023-02-15). "Video acceleration API (VA-API) now available on Windows!".
8. "Hardware/vaapi". wiki.libav.org.
9. (2 March 2017). "[Client] 2.2.6 Released – Significant CPU usage reductions included".
10. "datarhei Restreamer".
11. (29 August 2016). "Emby Server 3.0.6400 Released".
12. "Archived copy".
13. (3 June 2020). "Firefox on Fedora finally gets VA-API on Wayland".
14. "Firefox 80 Available With VA-API On X11, WebGL Parallel Shader Compile Support".
15. "Fluendo".
16. "Fluendo's New Codecs Support VDPAU, VA-API - Phoronix".
17. "Hardware Acceleration".
18. "H.264 VA-API GPU Video Acceleration For Flash - Phoronix".
19. "Hardware-accelerated video decoding, encoding and processing on Intel graphics through VA-API".
20. "Mailing list entry that describes uses of VA-API".
21. "RealPlayer for MID & Intel/Linux FAQ". HelixCommunity.org.
22. "Intel GPU".
23. "XBMC Gets Working Intel VA-API Support - Phoronix".
24. "This gist contains instructions on setting up FFmpeg and Libav to use VAAPI-based hardware accelerated encoding (on supported platforms) for H.264 (and H.265 on supported hardware) video formats.".
25. "gbeauchesne / mplayer-vaapi".
26. "Release Notes - 0.25 - MythTV Official Wiki".
27. "OBS Studio Now Supports VA-API For Video Encoding - Phoronix".
28. "OpenShot Library {{!}} libopenshot".
29. "VLC 1.1.0 release - VideoLAN".
30. "the xine project - News Feed".