The process is defined on the guide below (after it says: "To upgrade Drupal without drush"):
Please note that some of the commands described on that guide are intended for Linux. You will need to do the equivalent commands for Windows instead.
I hope this information is useful for you. If you have any other question, please do not hesitate to let us know.