Do you have to be a computer wiz or a math geek to pursue a career in video game design ?

No. What you do need is willingness to put in hard work.

Game design requires a lot of attention to detail. There are multiple areas of game design, including the artistic portions (making the games actually look good by creating images, sounds, and for some games, 3D models), game mechanics (deciding how hard a level should be, what you need to do to overcome an obstacle, if that obstacle should be seen once or on every level, those sorts of things), and programming (actually telling the computer how to make all of these pieces work together). Every one of them requires a lot of attention to detail, and often a lot of study in college or otherwise, to learn how to do it.

If you want to work on video games, you will have to decide which parts you want to learn about. As a programmer myself, I would recommend focusing on the programming side, because you can use the exact same skills you need to program games for a lot of other jobs, which gives you job security. The parts you focus on will tell you the skills that you need.

Also, while games will continue to advance, the skills involved remain largely the same. As a programmer, a lot of the same techniques go in to writing Pac-Man and games on modern video game consoles, it just takes a lot more programming to make a modern game. If you start small now, you will be able to grow to adapt to whatever super-advanced games are being made when you're out getting a job.

Do you have a sense of what part of video game design you want to work on? Do you want to be a generalist and be able to do a little bit of all of it?

No, you don´t, even when every computer related skill is an asset and will make you a better designer, you don´t need to be a computer/ math geek to be a game designer.
Since game design involves all activities related to determine how a game is going to be, its rules, its universe (plot, characters, environments, etc.) and finally is about defining the experience that the game is going to deliver to the players, you don´t have to be a genius on math, but you are going to need math sometimes.

Learn all the math you can, but in order to become a game designer also develop high comunication and social skills, you will need to transmit your ideas to many people from diferent disciplines (programmers, artists, producers, etc.) in a clear and efficient way. You´ll need properly writing to document and speech domain in a meeting while discussing about a new game.

Also develop your observation and analytical skills, play a lot of diferent games from every genre, but don´t leave you experience just like that, try to learn from every game you play, find out what´s working to add fun factor and what is not, if you can write that analisys way better!

Develop your creativity for new concepts with exercises, for instance write down new ideas to make slight variations of your favorites games, or if you are enjoying a movie, book or comic, try to come up with a game adaptation for that universe and write it.
Also very important is to take any course, talk, program, workshop related to game design, don´t hesitate on that.

Finally don´t worry if the games are getting more and more complex (usally AAA titles), sometimes is harder to design an original and fun but simple game, also simpler games obtain success and popularity and they will always have audience (in mobile market for intance).
It doesn´t matter if it´s a complex concept or a simple one, the groundbreaking blockbuster game of the future may be your idea, so if you want to become a game designer keep going!

I'd also take a step back and make sure we are all taking about the same thing. Game Designer is a specific job title in the field of video game development. The game designer is the person who comes up with the experience that is going to be created through a mix of story and mechanics. The programmers write the code and the artists supply the assets. Together, they make up the game development team.

So, based on what you asked, a game designer does not have to be a super math nerd, but he or she does need to be able to work with numbers in order to balance out the sides in a war game, decide how much a +2 sword is going to cost at the blacksmith shop, and how many experience points it takes to move up each level. Game designers spend long days with spreadsheets and databases. Additionally, it is very helpful for game designers to understand computer code and to be able to talk the talk with programmers. I hope this helps.

Also, not mentioned is do your research and figure out why would someone hire you as a designer? Are you creating some new, different original? So, to answer those questions, like they said above, work hard. Push your limits and plenty of free stuff online to learn different techniques and skills to best develop your passion. You've to REALLY love making games, not just playing them. You've to think outside the box, think behind the curtain in other words, how does it work and why. Look on many different kind of games, old and new, to see why they work and don't. If you're going to create games you need the basic framework behind it, the knowledge but also the passion to make something totally you. You can create games right now, free programs like Game Salad and Scirra, construct 2. Even play around in Unity but might be a bit much to start with. Use software, play around, Gimp, Photoshop,etc, you need to know the building blocks that are used for game development. Start with paper, create it in words and some basic line art or whatever, does it work for you? Show someone, why do I want to play this? Would I play it again? Research. Math will help of course but don't concentrate on that, concentrate on what is behind the creation of a game. Good luck.

One of the best ways to start is to download GameMaker:Studio or Unity (both have a free version - Unity is professional quality) and they have tutorials on how to get started. You will definitely need to know Math - but there's a lot of templates you can download so you can see how the coding is done, etc.... a great way to get started.