How much faster you ask? For a 16 bit Degrain3 script it’s 95% FASTER ON A 9800X3D!
It was done by cleaning up the code structure, using better algorithms, using new CPU instructions, optimizing everything and fixing really stupid bugs.
Go RTFM on GitHub or install it from PyPi (vapoursynth-mvutensils)
Stupid bugs
Let’s start with the most stupid bug of them all: The motion estimation code in Analyse generates 4 predictors, guesses on what the motion vector for the current block should be. These come from motion estimation at higher levels, neighboring blocks that already have known motion vectors and some other similar guesswork. Very often these are identical, especially in low motion video, where all 4 predictions can be (0,0). Instead of pruning the obvious duplicates the code would still test all 4 identical values and reach the same result. No longer checking the same motion vector over and over again gave an instant 7% speedup.
Did I say it generates 4 predictors? That’s not quite true, the code generated 5 but the 5th one was never used and now never will be.
Better algorithms
A common anti-pattern in the MVTools codebase is to allocate huge temporary buffers. Usually the size of a whole frame. Sometimes twice the size. In some cases 6 buffers the size of a frame. Apart from RAM being very expensive nowadays this has another major downside, the working set doesn’t fit in the CPU cache and as such you become memory bandwidth limited very quickly. What has been done in every filter is to instead process parts of the frame in tiles/chunks to greatly improve cache performance.
In the original MVTools most of the Flow* filters also need to reassemble the super clip into one large frame (the Super output doesn’t store the pel image as a single image, instead it’s stored as 2×2 or 4×4 images with the different offsets) which would take time and for pel=4 require a huge amount of memory. Now they all directly use the Super clip.
A lot of the C code was also modified to allow compilers to auto-vectorize it or generate better code in general.
New CPU instructions
New CPUs have AVX512 and many of its additions. These instructions can help quite a bit. For example the SAD16 code was sped up by 50% extra by using the VNNI instructions. I don’t believe anyone else has used this trick yet. And that was after the first speedup of 50% after using the applicable techniques from x264’s code. Also note that x264 has a much easier SAD calculation problem since not all 16 bits are actually used. Just like x265.
Optimizing everything
Avisynth-descended code usually shows another anti-pattern: The simple functions have intrinsics/assembler written for them while the more complicated ones don’t. This generally means that things the compiler could (sometimes with a small nudge) auto vectorize better than poorly thought out intrinsics become slower. And the really complicated functions where the compiler runs out of good ideas go untouched. Not here. Some functions like FlowBlur require AVX512 to make the optimization dream happen but every filter is optimized. It may take a week or two to think of a sane algorithm for a variable fetch memory gather bound filter but it’s worth it. I mean QTGMC uses it in like one place. Must be important.