back

ffmpeg & equirectangular panoramas
to cubemap and cubefaces and back

I've recently started to use the open source command line tool, "ffmpeg", version 4.3.1, in panoramic photography. Here with still image formats. But the real fun starts with video formats.

In ffmpeg (in stereo was where I found it), there is a frame size limit of ≈16000x16000 px over/under (or 32000x8000 side/side). I tried to work with 16384x16384 and it failed. Jpgs can give corrupt files, tif and png and many other picture and video formats works. – I'm learning ffmpeg as I go along – I found the "untile" filter when writing this (December 2020).



Example, equirectangular file.

The new filter "v360" converts 360° videos/images between various formats. Here I'm converting an equirectangular panorama to a cubemap in 6x1 format (horisontal).
ffmpeg v360 filter
v360 filter, cubemap face names: right 'r', left 'l', up 'u', down 'd', forward 'f', back 'b'. Default value is 'rludfb'. To get the forward cubemap face first or last in the cubemap, set order thus:
$ ffmpeg -i eq.tif -vf "v360=input=e:output=c6x1:out_forder=fbrlud" cubemap_fbrlud.tif

$ ffmpeg -i eq.tif -vf "v360=input=e:output=c6x1:out_forder=brludf" cubemap_brludf.tif



Example, cubemap_fbrlud.

I've added a set of 6 small (240x240 px) cubic faces, an equirectangular (960x480) pano to play with and a text file with a few commands (should work as is on Mac and Linux, with some adjustment on PC): zipped files



Example, forward cubic face: 5.

Untile the cubemap to it's six faces:
ffmpeg untile filter
$ ffmpeg -i folder/cubmap.tif -vf "untile=6x1" folder/%d.tif
or keep the "brlud" part of the cubemap intact, with a cubemap of 1440x240 px:
$ ffmpeg -i cubemap_brludf.tif -vf "crop=1200:240:0:0" cubemap_brlud.tif
To extract the forward cube face, with a cubemap of 1440x240 px. (Or use the untile filter.)
ffmpeg crop filter
$ ffmpeg -i cubemap.tif -vf "crop=240:240:960:0" face05f.tif

Reassembly of the cubemap.
ffmpeg tile filter
v360 filter, cubemap face names: right 'r', left 'l', up 'u', down 'd', forward 'f', back 'b'. Default value is 'rludfb'. Extract each cubemap and rename right to 1.tif, left to 2.tif etc. Then reassemble them.
$ ffmpeg -i folder/%d.tif -filter_complex "tile=6x1" folder/cubemap01.tif
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
This is not the same order that Adobe, Apple, Hugin, Pano2VR, Panotools, Quake II or RealViz has been using (frblud). WHY?!, so if you have cubemaps from them you have to untile and reassemble them to work with this filter (or use Pano2VR or similar). It seems to be the order of 'NVIDIA Texture Tools Exporter'. Blackmagic Fusion + KartaVR uses yet another order.
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ffmpeg's v360 filter can convert the cubemap back to equirectangular after reassembly:
$ ffmpeg -i cubemap01.tif -vf "v360=c6x1:e" equirectangular01.png
I notice that the forward cube face gets placed in the center of the equirectangular file. Like in the panorama on top.
To offset the forward cube face to the left in the equirectangular file, in degrees:
$ ffmpeg -i equirectangular01.png -vf "v360=input=e:output=e:yaw=90" equirectangular01_90.png
yaw offset pixels from one side to the other, like offset filter with horizontal wrap around in PhotoShop.



Example, equirectangular_90.

And here are more filters usable in panorama work.
To desaturate the image:
ffmpeg hue filter
$ ffmpeg -i pano.tif -vf "hue=s=0" pano_black_white.tif

Combining an image or panorama for the left eye and one for the right eye to stereo:
ffmpeg vstack filter
$ ffmpeg -i left.tif -i right.tif -filter_complex "vstack" left_above_right below.tif
ffmpeg hstack filter
$ ffmpeg -i left.tif -i right.tif -filter_complex "hstack" side_by_side.tif

And to make the anaglyph:
ffmpeg stereo3d filter
$ ffmpeg -i above_below.tif -vf "stereo3d=abl:arcd" abl_anaglyph.tif

$ ffmpeg -i side_by_side.tif -vf "stereo3d=sbsl:arcd" sbsl_anaglyph.tif

my very similar: "ffmpeg for stereo VRs" where you see the value of being able to handle separate cubic faces for animation/video.

See also:
FFmpeg_Book.pdf  from http://www.astro-electronic.de   regularly updated

And I find this free ffmpeg course to be valuable: https://slhck.info/ffmpeg-encoding-course

back