This is a small recorder I have made to record short messages for my child. 
Just turn it ON and hold the recording button for 2 seconds so that the recording LED indicator is activated. 

It is based on a Groove recorder and is powered by a 9V battery in a common 9V battery case (Locked with a small screw and glued in the slot). I recommend using a DC-DC converter (e.g. TSR 1-2450SM) to convert the 9V to 5V instead of going with a 7805 converter. This prolongs the battery life.  The micro switch is quite a generic size (Link here ). You will also need to do a bit of soldering to make it work. The microphone needs to be detached from the board and also the recording indicator LED should be detached and replaced with an external 5 mm LED and some R400 resistor. The new recording switch is then connected on the legs of the embedded switch. The board is held inside with a hot-inserted M2.5 screw. The volume has a huge range and the volume knob can be accessed with a screwdriver from the side.