I am currently working full time, but interested in doing freelance work.
I don't want to drop everything and jump right into it not knowing if this is what I want to do.
How can I go about finding my first client while keeping my day job?
This is always hard to find work at starting no matter how intelligent you are ... but you can get slightly easier if you have created any project to show , and contribute in some open source which would be good and you can tell about this either to client or to company(in case you want to do job )
and its easy to get clients by friends/family references
I got my first client by literally going door to door in local businesses introducing myself. It turned out that one of the business owners was an old family friend, and so I got a client. If I had it to do over again, I would probably start by making a list of friends who have businesses, and approaching them asking for feedback rather than work, and then see what I could do from there. You never know. One of them may want you to do something for them or they may know someone, but if not, at least you will likely get some constructive and local feedback.
This is part of the pattern of doing things for multiple reasons and being open to different kinds of benefits that may come from an activity rather than starting off and expecting that a single effort will make everything work.
As others have noted, personal and professional networks are the most common sources for a first client.
And although you haven't asked for this answer, I'll add that if you are employed full-time already, make sure you carefully review the terms of your employment contract before committing to any freelance work. Many employment contracts contain terms that make all intellectual property created by you (regardless of whether it is done on company-time or your own time) belong to your employer. In such circumstances, you could be putting yourself and your clients in a very difficult position by selling or licensing intellectual property created by you but owned by your full-time employer.
In my case, I got my first clients by my friends' references. Depending on what you want to do as a freelancer it's good idea to gain some experience and reputation. It's also very helpfull to enhance your portfolio - one of the most important things, when you want to get your clients from the internet, because it is your portfolio that represents you on-line.
And don't forget to post some ads on websites, that are freelancer-oriented and you may get a commission from and also on websites related on what you want to do.