An information processing device and method, a program, and an information processing system, for location identification at small granularity with high accuracy are disclosed. A portable terminal has a camera in addition to a GPS module. In a location recognition model construction phase, a location recognition model is constructed from an image sequence taken by the camera, associated with location information collected by the GPS module and the like, and stored. In a location identification phase, the current location in a global area is identified by the GPS module, and the current location in a local area is identified on the basis of the location recognition model selected from the identification result and the image sequence obtained at the time. The present invention can be applied to portable terminals such as notebook type personal computers, PDAs and mobile telephones.