Yes, add a button for say "PLAY IN VLC" or something and associate it with the "SKIN:MOVIES:PLAY MOVIE" event. Leave the other button assigned to "SKIN:MOVIES:PLAY MOVIE NEW"
define external video players under FILE TYPES. you'll need to get the command line parameters for VLC.
F2 (enter setup)
Click on FILE TYPES on the left
under DETAILS, click BROWSE beside program and browse to the VLC executable and click OPEN
Under EXTENSIONS, include what extentions you want to be used by this player (IFO and VIDEO_TS for dvd rips, avi, etc. NO PERIOD IS NEEDED, just the extension, separated by commas. for example ifo,video_ts,avi)
under COMMAND LINE SWITCHES, enter the command line switches needed to launch a movie in VLC from the DOS prompt or the RUN command (for example, in TheaterTek I use "%parameter%" /AP 2 /nosplash (Quotes are required around the parameter variable to handle file names/paths with spaces) %parameter% is from the database, it's the path to the movie file. The other ones won't apply to your situation, they are specific to TT
You can also enter MOVIES into the category if you want (or whatever your category is), but it's not necessary.
Then click ADD
The new program and information should now be in the list above, highlighted in YELLOW (this means it's disabled). put a CHECKMARK by enable to allow it to work.
Hope this helps