We use this expression to agree with an idea suggested by someone else.
Can we meet up at 8 o'clock, not 7 o'clock? I've got lots to do before we go out.
Fine by me.
Shall we order one large pizza and share it? It'll be cheaper.
Fine by me. I'm not too hungry.