Re: You’re invited for dinner but told to worry about bringing anything.

In answer to your question, I'd bring nothing as requested.
I assume (perfectly correctly) that I'd been invited because of my value as a person, and I can certainly understand that likelihood.

(Of course, it's nothing to do with me being a tight-fisted Yorkshireman.)

Re: You’re invited for dinner but told to worry about bringing anything.

I agree.
It’s nice having beautiful flowers after the event is over as a reminder of the fabulous evening with great friends.