I have a video where I ride my old Beemer over some whoops at a slow speed. On that trip, a repeat of an earlier one with a real dual sport, I went over rougher terrain than I did with the dualsport (a GSPD). I came to a conclusion after riding up some incredibly dusty trails, without extremely low tire pressure, on street tires. The flywheel is there for traction. It hadn't rained in months and it really was dusty. The old bike chugged up some pretty serious hills.
Let's say you're going on a terrain at a constant throttle position. You're maintaining a certain speed. And suddenly, you hit a sandy spot. The tires spin, it revs up, and you slow down, all with the same throttle position (and more or less, similar power output). The flywheel, and really, all of the engine internals, act in a way to resist that. The lighter it is, the more freely it wants to spin.
The lighter flywheel, over the weak surface, will slip and then slip much more than the heavy flywheel. The heavy flywheel will let the wheel speed up some, but not as much. Hopefully, by then you're on more grabby terrain and things settle back down without skipping a beat.
This is especially noticeable offroad, but also on road. If you start to slip the tire under power, the faster it wants to rev, the more dramatic the break in traction will be.
Newer motorcycles and cars often seem to come with very light flywheels. They're quicker off the line, lighter, and can eventually get to a point where traction control becomes more desirable. Flywheel effect is natural traction control.
There's no all-around win here. Some things are heavy for a reason. Lighter flywheel might get you a few fractions of a second at the quarter mile, but it may also mean you're more likely to hurt yourself in the rain, snow, dirt, or anywhere else. It may also be the difference between an impassable hill and one that you can just barely get over.