Need help with my server

I set a source dedicated server up at home for hl2dm, and i'm running it on linux. I finally got my keeper's sound plugin to work correctly without it crashing my server (no clue why it did that anyway), but I'm having one persistent issue: Other people are having trouble connecting to my server. When I start the server, it shows up in the steam list no problem, but then like a minute later, it switches to showing my internal IP and says "server is not responding," and nobody can connect. I have port 27015 forwarded on my router (TCP & UDP) like I'm supposed to, but still having this issue. (TKC) ME is helping me out with this, but we're still having trouble with it too. So far the only people able to connect to my server successfully are (TKC) ME, (tkc) Stewie_Chopsewy, and myself. My server also crashes and restarts when I try to change the map. Anybody got any suggestions?