"Going Down Town..." is the name of a famous song that everyone knows by heart and joins in to finish with you as soon as you say the first three words.
a famous quote from the China town film, also refers to the extent to which you are going to take them down a peg or two, generally assuming that chinatown is far away.