Advance user may wish to convert a macOS installer application into a ISO file for installing macOS on virtual machines like VMware or VirtualBox.

Command:
— hdiutil create -o /tmp/BigSur -size 13000m -volname BigSur -layout SPUD -fs HFS+J

(Big Sur .app is more than 12 GB so volume size is 13000m, if you have other macOS installer you can choose 8500m )

— hdiutil attach /tmp/BigSur.dmg -noverify -mountpoint /Volumes/BigSur

— sudo /Applications/Install macOS Big Sur.app/Contents/Resources/createinstallmedia –volume /Volumes/BigSur –nointeraction

— hdiutil detach /volumes/Install macOS Big Sur

— hdiutil convert /tmp/BigSur.dmg -format UDTO -o ~/Desktop/BigSur.cdr

— mv ~/Desktop/BigSur.cdr ~/Desktop/BigSur.iso

The macOS install packaged downloaded from Apple website and has .app extension. You can convert any .app macOS installer to .iso by using this command.