I keep reading American writers that write:
'That being said'
'That having been said'
'With that being said'.
No. The idiom is 'that said'. Two words: that's it.
Nothing else is required. I don't need a tense. I don't need an extra preposition or conjunction or any other word.
That said: that does it!