It's difficult to diagnose why you're having a hard time, Sargon. How would you say you come across? Direct and sexual? Gaijin-smash culturally insensitive? Are you emulating J-style in dress and manner? None of those are right or wrong. In my dress I was half-way Western and half-Japanese. Not super stylish but not clueless English teacher or tourist style either. Like to help but it's hard without seeing video or something of your approach.
Dr Johnson rumbles with the RawGod. And lives to regret it.