At one minute past midnight on October 1st, 1966, the gates of Spandau Prison in Berlin opened, and a thin, gaunt man in his early sixties stepped out into the darkness. Waiting for him at the gates was a woman in a tailored coat, a man in a suit, and a silence that had lasted twenty years. Margarete Speer, now sixty-one, watched her husband approach. She had been married to Albert Speer for thirty-eight years.

She had raised their six children alone for the last twenty-one. She had defended him in the press, refused to let their children visit him in prison, and built a life in the ruins of the one they had shared. He walked up to her, barely acknowledged her presence, and offered his hand. She shook it.

Twenty years earlier, on the day he had been sentenced, she had been forty-one, with the youngest of their children barely old enough to remember its father’s face. Now that youngest child was twenty-five, and the man who had once run Germany’s armaments industry was free. Margarete Speer did not embrace her husband. She did not cry.

She shook his hand at the gates of the prison that had held him for two decades, and that gesture, cold and formal, told the world everything it needed to know about the marriage of Albert and Margarete Speer. In the final week of April 1945, Margarete Speer was not in Berlin with her husband. She was at the Berghof complex on Obersalzberg, Hitler’s mountain retreat in the Bavarian Alps, with five of her six children. Her husband was elsewhere, first in Berlin, then in northern Germany, where he had attached himself to Karl Dönitz’s short-lived Flensburg government as Minister of Industry and Production.

The war was in its death throes. Hitler had killed himself on April 30th, and the Reich was collapsing in on itself. Margarete, forty-one years old, sat in the mountain complex with her children, waiting for the Americans to arrive. They did on May 4th, 1945.

The family avoided internment as non-combatants—no one arrested the wife and five small children of a Nazi minister—but the household they had known, with its cooks and servants and chauffeured cars, was finished. The Berghof was taken over by Allied forces. The family villa in Berlin had been destroyed in the bombing. Margarete packed what she could carry and moved the children to Heidelberg, her own birthplace, where the Speer family had owned a home at Schlosswolfsbrunnenweg since 1918.

What waited for them there was not the family villa of their imagination. The Speer house itself was occupied—by whom, the record does not say, only that it was unavailable. Margarete and the six children settled instead into an outhouse on the grounds of her father-in-law’s estate, a cramped outbuilding that would be their home for the next eight years. The transition was abrupt beyond measure.

They had grown up in the shadow of the Berghof, around Hitler and his entourage, with servants and chauffeurs and the absolute certainty of their own importance. Now they shared a few rooms in a stone outbuilding in post-war Germany, a country reduced to rubble and occupied by its enemies. Ten-year-old Albert Jr. began developing a severe stutter in the months after his father’s arrest.

He was the oldest, the one who had seen the most of the old life, and the one who would spend years trying to speak normally again. On May 23rd, 1945, British troops arrested Albert Speer at Glücksburg Castle near Flensburg, where he had been cooperating for a week with the US Strategic Bombing Survey. The arrest ended Margarete’s eighteen years as the wife of a senior official of the Reich. She was now the wife of a prisoner of war, a man who would soon be tried for crimes against humanity.

For the next eighteen months, her husband’s case dominated everything. From November 1945 through October 1946, Albert Speer stood trial at Nuremberg as one of twenty-four major defendants. On October 1st, 1946, the International Military Tribunal convicted him of war crimes and crimes against humanity for his use of forced labor, and sentenced him to twenty years in prison. On July 18th, 1947, he was transferred to Spandau Prison in Berlin and registered as prisoner number five.

Margarete was forty-one. The youngest of her children was five. Ahead of her stretched twenty years in which she would raise six children almost entirely alone, manage a household stripped of every privilege it had known, defend her husband’s reputation when she chose to, and decide on her own terms what her family’s relationship with a man in a Berlin prison cell would look like. Of all the people Albert Speer left behind in 1945, no one’s life was reshaped more by his choices than hers.

The story of that marriage had never been simple, and its complications went back nearly to the beginning. Margarete Weber met Albert Speer in the summer of 1922, when both were teenagers in Heidelberg. She was the daughter of a master joiner who employed about fifty workers and served on the town council. He was the son of an architect, part of a family with generations of standing in the city.

They married in Berlin on August 28th, 1928, when Margarete was twenty-two. She was not welcome in her in-laws’ home. Albert Speer’s mother considered the Webers socially inferior, and it took seven years before Margarete was first invited to stay at her in-laws’ house. That seven-year wait taught her, early in the marriage, exactly what she could expect from the Speer family: little welcome, and less warmth.

The Reich years made her the wife of one of the most powerful men in Germany and one of the most absent. Between 1934 and 1942, she had six children—Albert Jr. in 1934, Hilde in 1936, Fritz in 1937, Margret in 1938, Arnold in 1940, and Ernst in 1942. The regime awarded her a Cross of Honor for her fertility, and Hitler gave the family a villa on the Obersalzberg in 1935.

None of it changed the central fact of the marriage. Albert Speer was almost never there. As Margarete herself put it bluntly in interviews decades later, for all practical purposes, the children did not have a father. He visited Berchtesgaden rarely, and when he did, it was mostly because Hitler was in residence.

He took his holidays with Eva Braun and his ministry team, not with his wife. When she tried to join him on one of those trips, she was not invited. The marriage existed in name, in the household they had built, and in the children they had raised—or rather, that she had raised. After 1945, she became the only parent the children had.

In early 1954, eight years into Speer’s sentence, a reporter from Maclean’s magazine interviewed her in the outhouse at Wolfsbrunnenweg. She was smartly dressed in a white blouse and gray skirt, defending her husband to a foreign journalist with measured restraint. She refused to argue the case at Nuremberg. She did not deny that slave labor had been used in his factories.

She asked instead, quietly, whether eight years was not already enough. She had been raising six children alone since the surrender, she pointed out, and the arithmetic of her life was simple: she still had twelve more years of it ahead. One decision she made in those years defined her relationship with her husband as much as anything else. She refused to let any of the children visit him at Spandau.

They would form a wrong impression, she told the reporter, of a father they had hardly known anyway. The children only began visiting in their teenage years, and then only on their own initiative. Albert Jr. later said he saw his father in Spandau once a year, in visits initiated by the children, not by their mother.

Margarete had decided, years earlier, that her children would know their father through her telling, not through the bars of a prison cell. At midnight on October 1st, 1966, she waited at the gates of Spandau with Speer’s defense attorney, Hans Flachsner. Her husband walked out after twenty years, and he barely acknowledged her. They shook hands.

Within a few years, Speer began an affair with a German woman thirty years his junior, married to an Englishman living in London. Margarete and the children knew, and they said nothing. She remained at the Heidelberg house. He died in the woman’s company in London on September 1st, 1981.

Margarete outlived him by six years, dying at home on Christmas Day, 1987. She had held the household together for thirty-six years. After she was gone, nothing held the family together at all. The children who had inherited the heaviest surname of the twentieth century each found their own way through it.

The eldest, Albert Speer Jr. , had been born in Berlin on July 29th, 1934. By the time he turned three, his father was Hitler’s chief architect. He grew up in the outhouse at Wolfsbrunnenweg with his mother and siblings, finished school, and made the decision that would define the rest of his life: he would become an architect.

It was his father’s profession with a difference—Albert Friedrich Speer had practiced in Mannheim, and his great-grandfather before that, but young Albert Jr. was determined to take the longest possible route into the family trade. He worked first as a carpenter, beginning his apprenticeship while his father was still in Spandau, then enrolled at evening classes before entering architecture full-time at the Technical University of Munich in 1955. In 1964, he won his first international prize and opened his own office.

In 1977, he was appointed professor of urban planning at the University of Kaiserslautern, and in 1984 he founded Büro Albert Speer & Partner in Frankfurt am Main, which grew into one of Germany’s most respected urban planning firms. The contrast with his father’s work was deliberate at every level. Where Speer senior had designed at inhuman scale, monuments that made individuals feel small before the power of the state, Speer junior built around what he called human dimension—pedestrian-friendly streets, mid-rise density, ecological planning. He won few major commissions in Berlin, where his surname carried the heaviest weight, and built his reputation abroad.

His firm was lead designer for the central axis of the 2008 Beijing Olympics, master planner for Expo 2000 in Hanover, and contributed to Shanghai International Automobile City. It took part in Munich’s bid for the 2018 Winter Olympics and served as master planner for the Qatar 2022 FIFA World Cup. The irony was not lost on critics—the son of Hitler’s architect repeatedly worked for governments that drew comparisons to the regime he had spent his career trying to escape. In interviews, he was direct about the arithmetic of his life.

He told the Süddeutsche Zeitung that he had tried his entire life to separate himself from his father, and acknowledged that the man he had visited once a year in Spandau and who came out in 1966 was as foreign to him as one of his university professors. He told Reuters he had never considered changing his name. He was the eldest son of that father, and saw no reason to take another. The name had not helped him.

The stutter that had arrived when he was ten, in the months after his father’s arrest, stayed with him into his thirties. It only loosened its grip after a long bus trip across the United States in 1964, when he forced himself to speak to strangers in a language that was not his own. Albert Speer Jr. died on September 15th, 2017, in Frankfurt am Main, aged eighty-three, from complications following surgery after a fall at home.

His firm continues to operate. The second child, Hilde Speer, made a different choice with the name. Born on April 17th, 1936, she was nine when the war ended—old enough to remember Hitler’s hand on her shoulder in photographs at the Berghof, and young enough to be reshaped by everything that came after. In the summer of 1953, the United States refused her a visa.

Hilde was seventeen and had been awarded an American Field Service scholarship to study at Hastings High School on the Hudson, just outside New York City. The decision was reversed only after public protest, including offers of hospitality from American Jewish families who insisted she should come. She spent the year living with a family, Dr. Richard Day, a children’s doctor and psychologist, with the identity of her father kept officially secret in the locality.

Her father’s name had become a question that opened with no, and required others to argue for yes. In Heidelberg, she attended a school originally founded by Elisabeth von Thadden, a resistance figure executed by the regime in 1944. One of her teachers there, Dora Lux, was a Jewish historian who had survived the war in Berlin. Lux became one of the formative influences on Hilde’s life.

She would later write a biography of her, and shaped a question Hilde never stopped asking: what should the children of perpetrators do with what they had inherited? Politics came next. She joined the Greens during their breakthrough years in the 1980s and was elected to the Berlin House of Representatives, serving from 1985 to 1987 and again from 1989 to 1991. From 1989 to 1990, she served as the chamber’s vice president.

Her platform was consistent throughout—the peace movement, women’s rights, anti-racism, and a refusal to look away from what Germany had done. She married, raised children, became an education professor. Then, in 1994, she did something that defined her more than any office ever had. That year she inherited several paintings from her father’s collection.

Investigation suggested they had probably not been looted from Jewish owners, but she could not bring herself to keep them. She sold the paintings and used the proceeds to found the Zurückgeben Foundation—a German word meaning give back or return—which funded grants to Jewish women working in the arts and sciences in Germany. By 2019, it had supported more than 130 projects with around 500,000 euros. In the same year, the city of Berlin awarded her the Moses Mendelssohn Prize for her work promoting tolerance, and in 2019 she received the Obermayer German Jewish History Award.

She gave a phrase to the Associated Press that captured a position none of her siblings quite reached. She had always felt fortunate, she said, because she had known what her father was and what he had done very early. Many men and women of her generation had no such answers about their own families. She was still hosting refugees from Syria and Afghanistan in her home in Berlin well into her eighties.

The third child, Margret Speer, born on June 19th, 1938, named after her mother, made a third choice. In the 1990s, she was working as a photographer on the Berlin exhibition Topographie des Terrors, the permanent installation on the site of former Gestapo headquarters documenting the regime that had run it. She had spent the first six years of her life at Obersalzberg, a few hundred meters from the Berghof. She studied archaeology at Ruprecht-Karls-Universität, married the archaeologist Hans Nissen on April 14th, 1962, and lived in Baghdad from 1965 to 1967, then in Chicago, and finally in Berlin, where she taught herself photography and built a career in architectural work.

In 2004, she published her memoir, “Zinzi die Tochter Speer, Are You Speer’s Daughter? “, through Deutsche Verlags-Anstalt in Munich. The title was the question she had been asked her entire adult life—a question that never stopped. The book described a father who was egocentric, distant, and far more invested in constructing the image he left behind than in being present for the children who carried his name.

She named his crimes plainly. She did not seek to exonerate him. The fourth child, Arnold Speer, was born in 1940. His original name was Adolf, given in the year his father’s career reached its peak.

After the war, the family quietly renamed him Arnold, and the public record of his life is thin. By various accounts, he became a community doctor and stayed out of public view. Two of the six made a choice that left even less behind. Fritz Speer, born in 1937, and Ernst Speer, born in 1942, gave no interviews, wrote no memoirs, and made no public statements about their father.

In a family where the surname guaranteed a question every time it was spoken, silence was its own kind of answer. Six children inherited one of the heaviest surnames of the twentieth century, and each made a different choice with it. One designed cities for the modern world. One funded restitution for those their father’s regime had harmed.

One photographed the regime’s archives and named what her father had done. Three said almost nothing at all. After Margarete’s death in 1987, they lived in different cities, worked in different professions, and never came together as a family. Their mother had held the household together for thirty-six years.

After she was gone, even that connecting figure was lost.