This is a personal reference to what I consider a lifelong work of pilgrimage, but I'd say it's quite complete. Some places that have just been shot from the outside have been omitted (ie St. Pancras' Station).

SCOTLAND

• Black Rock Gorge (Evanton)
  • the rocky landscape where Harry faces the Hungarian Horntail in Goblet of Fire
• Clachaig Gully (Glencoe)
  • where Hagrid's Hut was placed from the third film and thereafter
• ✔ The Elephant House (Edinburgh)
  • cafe where JK Rowling used to write whilst she lived at her sister's
• Glenfinnan Viaduct (Lochaber)
  • part of a route from Fort William to Mallaig that's possible to take from May to October, here
  • the Viaduct is shown in the sequence wherein Harry and Ron fly on the Ford Anglia to Hogwarts
• Loch Shiel (Lochaber)
  • used as backdrop ("Hogwarts Lake")
• Steall Falls, Glen Nevis (Lochaber)
  • the location for the First Task of the Triwizard Tournament

NORTHERN ENGLAND

• Alnwick Castle (Alnwick, Northumberland)
  • accessible via East Coast line to Edinburgh - stop by Alnmouth, which is 4 miles away, and then takee bus (X18) or taxi
  • where the flying lessons & Ford Anglia crash scenes were filmed
• ✔ Durham Cathedral (Durham, County Durham)
  • ...there's no explaining this one. <3
  • (but to whoever is reading this list and wants to visit, the Cloisters and the Chapter House were used in various scenes in Philosopher's Stone and Chamber of Secrets. The Cathedral is in the centre of town, in the Bailey area just outside the Castle, and is very easy to access)
• Goathland Railway Station (Goathland, North Yorkshire)
  • set for Hogsmeade Station
• ✔ Pedestrian bridge, York station (York, Yorkshire)
  • where Hagrid hands Harry his ticket to board the Hogwarts Express for the first time
• Malham Cove (Malham, North Yorkshire)
  • the rock formation where Harry and Hermione apparate without Ron on the search for Horcruxes

MIDLANDS

• Gloucester Cathedral (Gloucester, Gloucestershire)
  • the cloisters and the Monks' lavatory were used as sets for Philosopher's Stone, Chamber of Secrets and Half Blood Prince
• Hardwick Hall (Derbyshire)
  • used as inspiration for Malfoy Manor in Deathly Hallows Part One
• JK Rowling's childhood home (Coleford Road, Tutshill, Gloucestershire)
  • just thought I'd pay homage sometime
• The Forest of Dean (Gloucestershire)
  • the actual forest where Hermione and Harry apparated whilst searching for Horcruxes

OXFORD

• Blenheim Palace
  • where Harry sees a fragment of Snape's past wherein he's being harassed by the Marauders in the grounds
• ✔ Bodleian Library, Oxford University (Broad St.)
  • location for the Restricted Section scenes in Philosopher's Stone
  • open 9AM-7PM weekdays, 10AM-4PM Saturday - ticket office is at the Great Gate to the Bodleian Library, Catte Street
• Christ Church College, Oxford University (St. Aldate's)
  • various scenes were filmed at the grand marble staircase at the entrance, and the Great Hall is inspired by their dining hall
• ✔ Divinity School, Oxford University (Broad St.)
  • location for the Hospital Wing in many of the films
  • open 9AM-5PM weekdays, 11AM-4:30PM weekends
• ✔ New College, Oxford University (Holywell Street)
  • featured in Goblet of Fire, where Mad-Eye confronts Malfoy

EAST ANGLIA

• Godric's Hollow (Lavenham, Suffolk)
  • the now-sold De Vere House was chosen as Godric's Hollow post-_Philosopher's Stone_

SOUTHERN ENGLAND

• Black Park (Wexham, Buckinghamshire)
  • take a train to Langley Station and walk 2-4 miles on the back roads and country park trails (no footpath available)
• Burnham Beeches (Buckinghamshire)
  • where Ron gets splinched
• Dursley House (Bracknell, Berkshire)
  • 12 Picket Post Close, Winkfield Row - of course it's not possible to go inside as it's a residential area
• Frithsden Beeches (Chiltern Hills, Berkhamsted, Hertfordshire)
  • featured in either Goblet of Fire or Prisoner of Azkaban. Look out for the Queen Beech, or "Harry Potter Tree"
• Lacock Abbey (Lacock, Wiltshire)
  • the Cloisters, the Chapter Room and Sacristy were used in various scenes of Chamber of Secrets and Half-Blood Prince
  • other Harry Potter places were filmed in the village of Lacock, such as Slughorn's House, Budleigh Babberton Square and Godric's Hollow (_Philosopher's Stone_ only)
  • walk 3 miles from Melksham Station
• ✔ Leavesden Studios (Watford, Hertfordshire)
  • ...of course.
• Swinley Forest (Bracknell, Berkshire)
  • where Hermione stuns Harry and they're chased by Snatchers just before going to Malfoy Manor
• Virginia Water Lake, Windsor Great Park (Virginia Water, Surrey)
  • where Harry fought the Dementors in Prisoner of Azkaban and where Hermione speaks on behalf of Ron in Goblet of Fire

LONDON

• Australia House (Strand)
  • Gringotts Bank
  • closest tube station: Temple
• Chez Michele Florist, Borough Market
  • entrance to the Leaky Cauldron in Prisoner of Azkaban
  • closest tube station: London Bridge
• Claremont Square, Islington
  • where exterior shots of Grimmauld Place were filmed
  • closest tube station: Angel
• Harrow School
  • the Fourth Form Room was used as Professor Flitwick's classroom in Philosopher's Stone
  • the quickest way there is to take the tube to Harrow-on-the-Hill Station
• Heathgate, Hampstead Garden Suburb
  • where Hermione's house is located
• ✔ King's Cross Station
  • where the Hogwarts Express appears in Platform 9 3/4, which is actually filmed between platforms 4 and 6
• ✔ Millennium Bridge
  • wrecked by Dementors in Half Blood Prince
• Reptile House, London Zoo
  • scenes with the python in Philosopher's Stone
  • closest tube stations: Regent's Park (20 min. walk), Camden Town (15 min. talk)
• Scotland Place, Great Scotland Yard
  • entrance to the Ministry of Magic
  • closest tube station: Charing Cross
• Surbiton railway station
  • railway and cafe scene at the start of Half-Blood Prince
• The Glass House, Leadenhall Market
  • original entrance to The Leaky Cauldron. It used to be an empty place before The Glass House was built
  • closest tube station: Monument

IRELAND & WALES

• Cliffs of Moher (Galway, County Clare, Ireland)
  • where Dumbledore takes Harry to find the Locket
• Freshwater West Beach (Pembrokeshire, Wales)
  • where Shell Cottage was located
• Lemon Rock (Iveragh Peninsula, County Kerry, Ireland)
  • where Dumbledore stands with Harry before the cave (added with CGI)