x264 Encoding Options for Hardware Compatibility & DXVA


This thread is a spin-off of some posts regarding how to get MKV files encoded with x264 to work with hardware acceleration (a/k/a DirectX Video Acceleration, or DXVA) on ATI HD2000, HD3000 and NVIDIA 8000 video cards.

The following settings will ensure hardware compatibility with players like the PS3, and also allow smooth DXVA playback on ATI and NVIDIA graphics cards when using the Cyberlink H.264 decoder included with PowerDVD 7.

The settings have been tested to work with PowerDVD, ZoomPlayer, Media Player Classic – Home Cinema, using the Haali Splitter or the Nero Digital Parser, and using the Overlay Mixer, VMR7, and VMR9 Windowless/Renderless video renderers.

The Bottom Line
Everybody should be encoding HD content (1080p, 720p) to Profile High @ Level 4.1. For smooth playback, “–level 4.1” should be used to mark the file as compatible when encoding. The best x264 profile is the PS3 profile, except with B-pyramids turned off.

The PS3 profile options, sans B-Pyramid:

–level 4.1 –ref 3 –mixed-refs –bframes 3 –b-rdo –bime –weightb –direct auto –subme 6 –trellis 1 –analyse p8x8,b8x8,i4x4,i8x8 –8x8dct –vbv-maxrate 25000 –me umh –merange 12

Everybody should be encoding SD content (576p, 480p, or less) to Profile High @ Level 3.1 For smooth playback, “–level 3.1” should be used to mark the file as compatible when encoding. The best x264 profile is the PS3 profile, except with B-pyramids turned off and the level set to 3.1.

The PS3 profile options, sans B-Pyramid with L3.1:

–level 3.1 –ref 3 –mixed-refs –bframes 3 –b-rdo –bime –weightb –direct auto –subme 6 –trellis 1 –analyse p8x8,b8x8,i4x4,i8x8 –8x8dct –vbv-maxrate 25000 –me umh –merange 12

The Details
The associations responsible for the HD DVD and Blu-ray Disc standards decided the approved H.264 profile/level would be High@L4.1 for HD content, and so the hardware decoder manufacturers have built to that specification. The hardware decoder manufacturers have also decided the approved H.264 profile/level would be High@L3.1 for SD content.

That has resulted in two implications:

1. H.264 video must comply with the L4.1 and L3.1 limitations for the Decoded Picture Buffer (DPB) size, which limits the video resolution and number of reference frames.

2. H.264 streams must declare L4.1 or L3.1 in the “level_idc” flag (set by –level with x264) in order to achieve smooth DXVA playback.

The Settings
Keep in mind that, no matter the resolution, it is still generally recommended to only use between 1-5 reference frames, usually centering on 3.

As of the 2008-01-06 x264 snapshot, (core: 57 version: svn-717M) the following settings apply. There is a patch available that changes the way –ref functions when encoding, and how the DPB size is calculated. The patch hasn’t yet made its way into the popular x264 builds, or even the latest x264 snapshot, so unless you roll your own, you’ll have to use these settings. I’ll update the post when the patch is released in a build in the wild.

The DPB size is the max at the given resolution, so if your resolution is between the resolutions given, use the lower DPB size (i.e., 1920×816 is between 1920×720 and 1920×864, so you can have a max DPB of 5).

Resolution: Max DPB Size (x264 encoder settings)
L4.1
1920×1088: 4 (–ref 3 –bframes 3 –b-adapt –no-b-pyramid)
1920×864: 5 (–ref 4 –bframes 3 –b-adapt –no-b-pyramid)
1920×720: 6 (–ref 5 –bframes 3 –b-adapt –no-b-pyramid)

L4.1
1280×720: 9 (–ref 8 –bframes 3 –b-adapt –no-b-pyramid)
1280×648: 10 (–ref 9 –bframes 3 –b-adapt –no-b-pyramid)
1280×588: 11 (–ref 10 –bframes 3 –b-adapt –no-b-pyramid)
1280×540: 12 (–ref 11 –bframes 3 –b-adapt –no-b-pyramid)
1280×498: 13 (–ref 12 –bframes 3 –b-adapt –no-b-pyramid)
1280×462: 14 (–ref 13 –bframes 3 –b-adapt –no-b-pyramid)
1280×432: 15 (–ref 14 –bframes 3 –b-adapt –no-b-pyramid)
1280×405: 16 (–ref 15 –bframes 3 –b-adapt –no-b-pyramid)

L3.1
720x(any): 8 (–ref 10 –bframes 3 –b-adapt –no-b-pyramid)
720×576: 11 (–ref 10 –bframes 3 –b-adapt –no-b-pyramid)
720×480: 13 (–ref 12 –bframes 3 –b-adapt –no-b-pyramid)

Retrofitting MKVs or other H.264 videos
Existing MKV, MP4, TS or AVI files that work with DXVA, but do not play back smoothly, can have their level_idc value changed, if the H.264 video stream actually complies with Profile High@L4.1 (HD) or L3.1 (SD).

This process is for MKV files and has been adapted from a process originally described by DJBlu.

Download the following programs:

MKVtoolnix
MKVExtractGUI
H264info
Optional: AVInaptic

Extract MKVtoolnix into a folder.
Extract MKVExtractGUI into the same folder.
Extract H264info into another folder (can be the same folder).
Optional: Extract AVInaptic into another folder (can be the same folder).

Open your MKV with MKVExtractGUI. Check the boxes for all the video, audio, and subtitle tracks you want to keep and click Extract.

Once all are extracted, you will have separate files for your video, audio and subtitle tracks, plus TimeCode_TrackX.txt files for each track.

Open H264info.

Set Input = Extracted .h264 file
Set Output = new_file_name.h264

In the “Level (1 – 5.1)” box, enter 4.1 for an HD stream, or 3.1 for an SD stream.

Click Start and wait for the stream to be fixed.

Open the MKVtoolnix mkvmerge GUI (mmg.exe).

Click add and select the new_file_name.h264 file created by H264info.

Click add and select a file for each audio and subtitle track you need to add.

In the Tracks box, click on the “MPEG-4 part 10 ES” video track.

At the far right of the Timecodes box at the bottom of the window, click Browse and select the TimeCode_TrackX.txt file for your video track (probably Track1).

Click the “Format specific options” tab.

In the FPS box, enter the frames per second of your original MKV file. If you don’t know the framerate, you can use the AVInaptic tool to open the original MKV file to get all manner of information, including the FPS. If your video is FILM content, the likely choice is “24000/1001”. If your video is progressive NTSC VIDEO, the likely choice is “60000/1001”. If your video is interlaced NTSC VIDEO, the likely choice is “30000/1001”. If your video is PAL VIDEO, the likely choice is “25”.

Click on your audio track in the Tracks box.

Click on the “General track options” tab.

At the far right of the Timecodes box at the bottom of the window, click Browse and select the TimeCode_TrackX.txt file for your audio track (probably Track2). Do the same for any subsequent tracks.

Create an “Output filename” in the box at the far bottom of the window and click “Start muxing”.

Wait for the new MKV to be remuxed and you should have an MKV file with smooth DXVA playback.

 

 

 

source

 

Advertisements