I would think a tight larger size 2800 would be fine
I did some slow trail riding with my 3200 and it worked fine but I don't think I would have wanted to take it rock crawling

 

I was thinking 2200-2500.

 

An 80mm turbo is gonna be too big.

I think a Maggie would be perfect for you. It's not in boost until you want it to be with the way the vacuum blowoff works, but when you want it it's literally instantaneous.

The 80mm turbo will keep you out of boost when you don't want it, but it'll take forever to get boost with the tight converter when you DO want it.