Well, good old Winamp has this feature. You can add your whole library, browse it through Winamp, and delete files directly from it.

To be honest though, I haven't used it in ages, so I don't know if the latest build (5.8) has it still.

I think you're doing a lot of unnecessary work by running your windows maximized. Resize your Media Player and Explorer windows so both are visible at the same time and not overlapping. Some versions of Windows have a "window snapping" feature that will let you easily dock one application to the left half of your screen and another to the right side.

When both programs are on-screen at the same time, you can process a folder quickly, without even needing to touch the mouse. In the Explorer window, press Enter to play the currently-selected file in WMP. Switch back to Explorer with Alt+Tab. Delete the song with Delete, or move to the next song with Down Arrow -> Enter. Use the left hand for Alt+Tab and the right hand for Enter, Delete, and Down Arrow. I've done this sort of thing before for sorting through photos, and I very quickly got into a rhythm (yay muscle memory) that let me burn through a large collection in no time. There's still some processing overhead for each file, but you should be able to do all of that while you're listening to the current file and deciding whether to keep it or not.

If it won't let you delete a file that's still playing, you may have to start the next one and then go back and delete it.

EDIT: I no longer recommend this answer. See my other answer involving MusicBee, which combines the functionality of MPC-HC with the user experience of Windows Media Player, and then some.

I found a way to do this with MPC-HC, although I still would have preferred to have used Windows Media Player, which has advantages over MPC-HC such as music visualisations and a much more natural shuffle feature.

Disappointingly, even MPC-HC can only do this by using its playlists feature, which is enabled by opening the Playlists window pane. This is disappointing, but is so far the only way I've found to do this (incidentally, setting up a playlist is also a requirement to shuffle music in MPC-HC).

To delete files from within MPC-HC

1. Right-click on the folder containing your music files, and click the Add to MPC-HC Playlist context menu item (note that the context menu item needs to be selected on a folder, as from my testing adding a large number of individual files to an MPC-HC playlist results in a fatal crash)




2. Within MPC-HC, select View > Playlist, or use the shortcut Ctrl+7 to open the Playlists window pane.




3. All of the music files in the folder should be loaded into this pane. Right-click anywhere in it for a context menu with more options. Select Move to Recycle Bin to delete the currently playing track and move to the next one. You can also select Shuffle here to enable MPC-HC's shuffle ability.

Due to the playlist requirement of doing this in MPC-HC, I'm still open to other solutions involving MPC-HC, Windows Media Player or different music players entirely that does not require the use of playlists as a requirement. Visualisations would also be a huge bonus of any such solution.

There's an alternative hack that might work: use the sidebar to rate the songs as 1 star, then at the end, sort by stars in Explorer and delete the items with one star.

Note that you'll need to not have any one-star songs before you begin, or you'll delete them too.

Since kicken's answer to this question only seems to work on Windows 10, making me unable to test it, and since I got tired of trying to make MPC-HC work for this purpose (it does, but it's glitchy, its playlist view is plain ugly, and development on it has also been stalled for a while), I decided to spend the last 24 or so hours looking for and experimenting with an alternative music player.

I quickly realised I was no longer looking for just a music player - whereas previously I'd used my music player almost exclusively to listen to music, it was now overdue for me to move with the times and start using something that had more powerful music organisation abilities.

A much better alternative

I began by looking into foobar2000 and Clementine, based on recommendations from answers to this question and elsewhere, and although I found both okay, both were still lacking in comparison to the solution I tried last.

MusicBee is something I've been looking for for a long time without realising it - a modern music player and manager with an interface and user experience as beautiful as it is powerful. Ironically, I'd already had it installed because I'd checked it when last running Ninite, but I'd never started it up or thought to use it as my regular music player.

The one thing it is missing is a context menu item to add files in a folder to the Library, which I found odd, but clicking the Menu, Add Files to Library, and then navigating to the Playlists tab gives me the view of my files that I need:

From here, it allows me to play files and delete them as they're playing - both to the Recycle Bin and permanently (with built-in hotkeys for both).

Another thing that sets MusicBee apart from the competitors I tried is just how customisable it is - the Options panel is chock-full of settings to be enabled/ disabled and panels to be added/removed, from changing the program's icon (with four included options):

...to changing virtually every component of the layout:

It also comes with visualisations, as well as built-in transcoders and skins (many of them dark, for those who are into that sort of thing), and many other powerful features.

Then there's the Theater mode, which is a big sell for me since I hadn't until today found a full-screen music player view that rivals that of (Windows 7's version of) Windows Media Player:

Finally, MusicBee is complete freeware. That's worth mentioning because at this point this answer is starting to sound like a product plug, which I apologise for, but it's very rare that I come across a product that I know will massively transform the way I do things and make me ten times more productive as a result.

PS. Please don't judge me for my music tastes, this is just one of many collections of music that I've inherited and am trying to separate the wheat from

Add the songs to a playlist, save the playlist as m3u, and feed it to rm.

Besides all the great answers that suggest using software such as foobar, you could make a pretty sweet shell script that iterates through the music in a given folder, plays each one, then uses bash's built-in select to give you menu options:

1. delete this file


2. delete all .mp3 files in this folder


3. do nothing and proceed to next song

$ cat ~/bin/music_delete
#!/bin/sh
# usage ./music_delete my_folder
cd $1;
for f in *.mp3
do
 echo "Playing $f; press Ctrl-C once you've heard enough";
 mpg123 $f; # play the file
 select option in "Delete" "Delete All" "Skip"
 do
 case $option in
 "Delete")
 echo "Deleting $f";
 rm -f $f;
 break;
 ;;
 "Delete All")
 echo "Deleting all *.mp3 files in $1";
 rm -f *.mp3;
 exit 0;
 break;
 ;;
 "Skip"|*)
 echo "Skipping $f";
 break;
 ;;
 esac
 break;
 done
done

Example usage:

$ music_delete test2/
Playing test1.mp3; press Ctrl-C once you've heard enough
...
1) Delete
2) Delete All
3) Skip
#? 1
Deleting test1.mp3
Playing test2.mp3; press Ctrl-C once you've heard enough
...
1) Delete
2) Delete All
3) Skip
#? 3
Skipping test2.mp3
Playing test3.mp3; press Ctrl-C once you've heard enough
...
1) Delete
2) Delete All
3) Skip
#? 2
Deleting all *.mp3 files in test2/

Advantages:

• Everything can be done from the keyboard with no pesky clicking. Even if you use a GUI program to play the file you can exit it with Alt-F4 or Ctrl-W


• Should work on almost any machine (Posix compliant so you don't even need bash technically)


• Using mpg123 or other command-line utils to play the music files, you can skip back and forth through the song easily with just the keyboard


• Leet af


• This approach is extensible to any type of file you want to review, including videos, pdfs, pictures, etc.


• Can change the script to use GNU find and recursively search through whole directory structures for files.

Caveats:

• Need a posix-compliant shell installed. On Windows this can be installed with e.g. the Linux subsytem or Cygwin


• Need the mpg123 executable installed and in your user's path (which Cygwin can install as can Ubuntu or other Linux distros


• 
  mpg123 can be replaced with any program that plays mp3 files and accepts them as command line arguments. This should include foobar and vlc to name a few


• You will need to put the full path to the executable if it's not in your user's path


• You can change *.mp3 to a list of file formats e.g. *.mp3 *.flac *.alac
  


• I haven't tested this with any files with weird names, even just names with spaces, so be careful and run some tests!

Both Winamp and Clementine can delete files while playing. Clementine is now better than Winamp, and is still being updated regularly. Windows Media Player makes an awful music player.

I suggest using Windows File Explorer instead of WMP to do the finding and deleting of files. These would be the steps.

1. Create a new folder for the files you want to keep.


2. Open two windows of Windows File Explorer. One will point to the existing music folder and the other will point to the new one.


3. Highlight any file in the existing folder. Play the song (press Enter, right click and select Play with Windows Media Player, double click, etc). Then close the music playing software (Media Player, Groove, whatever).


4. If you decide to keep the file, cut and paste it to the new folder. Otherwise, delete it.

The advantage of this approach is that the original folder will only contain files that have require decisions. You don't have to keep track of anything, and you don't really have to do much searching.