Good news is it can be enabled for when it is really needed but should be noted that yes if you have proper E2EE you cannot have things like cloud recording or include telephone calls as well. If those were possible then the encryption is broken on the server side (like many video conferencing tools actually do where it is actually decrypted at the server side).
Remember that any service which can offer phone breakouts, do cloud recording, reset your lost password, etc has not got full E2EE in place. Their server side is intercepting data in the clear before encrypting it on its next leg to other participants.
More details at Zoom finally adds end-to-end encryption for all, for free – though there are caveats
Perhaps we can talk about extra security and compromises honestly