It's a common position of moderates that you can’t define what God is (e.g. Karen Armstrong near 17:10 in this interview).

It seems plausible and consistent for a non-believer to say the gods are undefinable or ill-defined, or to define gods that they don't believe in. But I can think of at least three problems when a believer claims that God can't be defined (apparently this is an "apophatic" theology):

  1. The statement self defeats. That "you can't define God" is itself an attempt at a definition. It's akin to a relativist argument, and suffers from the weaknesses of such arguments: they either attack themselves, or else exempt themselves from their own attack - inexplicably.
  2. How do you know? Unless you have a formal proof of unknowability, you're just talking about something that you admit not knowing the half of. That's frowned on, for God's sake.
  3. How do you know, then, that he's worth worshipping? Given all that 'mystery', he might actually be much less appealing than you think he is.

Defining terms, with all its difficulties, is a basic goal of conversation. With moderates, it can feel deeply pointless; God is the topic they insist on, but they withhold all possibility of agreeing terms. Fundamentalists, of course, are happy to define God for you; and with no hangups.

(Added: Daylight Atheism has a better post than this, on the same subject.)