My question is, if i get an additional 3090, would it be better than 2x3080ti + 3090?
Working with two similar GPUs would be easier, but having any of the two configs be it 2x3080ti + 3090 or dual 3090 is just a luxury if you aim to learn ML or implement some models.
The majority of people learning ML either for academic, job transition, or passion, have been working with hardware that's far less powerful than any of the GPUs mentioned here. Just a few months back
Vesuvius Challenge's first stage was achieved with mere GTX 1070.
ML is not gaming, a better setup won't yield better performance or results rather it's your proficiency with statistics and programming.
Aside from that to answer another question asked above, RTX 3060 is more than enough to implement the majority of ML models. When Meta's Llama model was leaked and later unveiled to the wider masses, people were racing to figure out newer quantization methods to run the models on their Macs, PCs, and even smartphones XD.
As for another question regarding NVlink, it is useful cause it decreases multiple calls to the CPU, provides more granular GPU memory management, and ofc more bandwidth, these factors can be taken advantage of while prototyping. If you don't see the benefit in that then it's not useful in a meaningful way for your work.
If you purely want to just work/play with LLMs and plan to buy a new GPU or upgrade your current setup then going for the newer 40xx series is a good bet. Other than that thinking of RTX 40xx having a major benefit over 30xx series just because of fp8 is moot. You can always modify the model to play nicely with fp16 and produce stable results.
Also, while training your model, the data is distributed into batches and sent to each card to be processed. So the vram does not add up.
Having similar GPUs helps in ascertaining the memory allocation with different batch sizes and circumventing oom errors early on. One thing I have done recently is to use a spare rx550 to drive my monitor to keep the 3090s solely for running the projects. Also, Radeon helps with the out of box support in Linux.