What is most striking about the retrospective of Edwin Smith’s photographs at the Royal Institute of British Architects (until 6 December) is his ability to capture the human relationship to buildings. A woman’s silhouette in a shadowy side street is dwarfed by York Minster; a man in a suit casts a short shadow before the long shadows of the palatial façade of the Royal Exchange; a cat lingers uncertainly in the gateway to Ampton Hall.

Born in Islington in 1912, Smith got his first camera by saving up Cornflakes coupons. It was a Kodak Brownie, named after Palmer Cox’s cartoon sprites and not much more than a cardboard box with a lens. In 1930 Smith won a place at the Architectural Association School of Architecture, but had to withdraw two years later when his family ran out of money. He found a job as an architectural draughtsman and continued to take photographs, though he saw both as secondary to his real ambition, painting. Enid Marx gave him a better camera and showed his pictures to her old RCA tutor, Paul Nash, who in turn introduced Smith to the editor of Vogue and arranged for him to use the darkroom at Lund Humphries for free. Smith didn’t like fashion photography, but it paid the bills while he continued to paint.

Regular commissions arrived from the Stuart Advertising Agency. Smith experimented with new cameras and darkroom techniques: processing was as important as capturing the shot. In his photographs of Fylingdales he added a sun to the grey horizon by placing a coin on the photographic paper as it exposed. Although he continued to call himself a painter, he was animated by photography’s power to ‘set the visible world upside down, to decapitate your friends, haunt their houses and distort their wives’, as he wrote in All the Photo Tricks.

The two greatest influences on his work arrived in swift succession: first, the discovery of Eugène Atget, whose unassuming style and portrayals of popular culture inspired Smith to seek out subjects in the music hall, the pub, the circus and fairground. Then, in 1943, Smith left his wife and son for Olive Cook, who worked with Kenneth Clark at the National Gallery and had participated in the Recording Britain project, which commissioned artists to document threatened buildings and landscapes. Their interests convened around Englishness, eccentricity, disappearing worlds, architecture – she encouraged him to look at buildings – and from 1944 to 1975 they contributed to the Saturday Book, Leonard Russell’s annual miscellany, which was defiantly unconcerned with ‘work, problems, issues, movements, trends’ and instead celebrated butterflies, toy theatres (Smith collected them), Pears soap, Elizabeth Bowen’s confessions and an anthology of waistcoats. Writing on matchbox design, Smith said that in only a few such things did the unsophisticated allure of popular art ‘inextricably linger’.

The postwar move from modernism to an appreciation of traditional materials and what Robert Elwall, in his monograph on Smith, calls ‘a reassertion of the picturesque principle’ led to a series of illustrated books about British architecture and landscape. Smith’s first assignment for the newly established Thames and Hudson was English Parish Churches; he called the experience ‘among the most vivid and poignant of my life’. His photographs were much admired for their richness and humanity, and for restoring to architectural photography its ‘chromatic value’ (Betjeman said they were ‘full of the suggestion of colour’). Unlike the shadows and stark white of recent years, Smith’s photographs were full of textural greys: the play of light on dry stone walls, thatch, marble and foliage.

In later years he travelled, and his photographs became grander, but his subjects continued to be in conversation with their environment. In The Garden of Schloss Benrath (1961) a girl in a white dress crosses the lawn in front of the statue of young prince, who seems to look directly at her: we look from him to her. In Lion at Porta Regia (1963) the strangely pitiful sculpture of a lion in the foreground appears to utter a last gasp for the defeated old man sitting by a column in the background.

Other images appear too good – or too contrived – to be true: a frowning peasant, in House in Urbino, framed by a black window as she looks out over hanging laundry, striated hills and buildings rising up behind; a lone nun on the Giant’s Causeway (1965); two boys on a doorstep in Kilkenny (1965), one in a white shirt, the other in a dark jumper, echoing the colours of the two doors behind them (are they really laughing?).

The RIBA has 60,000 of Smith’s negatives and a new gallery space in which to showcase them. The curators have, naturally enough, focused on his architectural photos but also given space to his playful side, and been playful themselves, setting Smith’s Florence Shop Window (1963) – models of shy Greek nudes losing their robes – next to Atget’s 1923 shop window, Men’s Fashions Paris, all stiff male mannequins and pressed trousers.