At midnight on October 1st, 1966, Margarete Speer stood at the gates of Spandau Prison and waited. Beside her was Hans Flachsner, her husband’s defense attorney. For twenty years she had raised six children alone, defended a man she could not visit, and watched her family scatter across continents and convictions. Now the gates opened, and Albert Speer walked out.

He barely acknowledged her. The two of them shook hands, as though they were strangers meeting for the first time, and the reunion that should have closed two decades of waiting lasted only as long as a single gesture. Margarete Speer was forty-one years old when her husband was sentenced to twenty years in Spandau. She was sixty-one when he came back.

In the years between, she had become the only parent her children knew, managing a household stripped of every privilege it had known and making decisions on her own terms about what her family’s relationship with a man in a Berlin prison cell would look like. On the final week of April 1945, she was at the Berghof complex on Obersalzberg with five of her six children. Her husband was elsewhere, first in Berlin, then in northern Germany, where he had attached himself to Karl Dönitz’s short-lived Flensburg government as Minister of Industry and Production. The children ranged in age from ten-year-old Albert Jr.

down to Arnold, who was four, and Ernst, who was two. American forces reached the area on May 4th, 1945. The family avoided internment as non-combatants, but the household they had known was finished. On May 23rd, 1945, British troops arrested Albert Speer at Glücksburg Castle near Flensburg, where he had been cooperating for a week with the U.

S. Strategic Bombing Survey. That arrest ended Margarete’s eighteen years as the wife of a senior official of the Reich. With the Berghof under Allied control and the family villa in Berlin destroyed, she packed what could be carried and moved the children to Heidelberg, her own birthplace, the city where the Speer family had owned a home at Schlosswolfsbrunnenweg since 1918.

What waited for them was not the family villa they had imagined. The Speer house itself was occupied. Margarete and the six children settled into an outhouse on the grounds of her father-in-law’s estate, where they would live in cramped conditions for the next eight years. The transition was abrupt.

They had grown up around Hitler and his entourage, with cooks and servants and chauffeured cars. They were now sharing a few rooms in an outbuilding in post-war Germany. In the months after his father’s arrest, ten-year-old Albert Jr. began developing a severe stutter.

Their father’s case dominated the next eighteen months. From November 1945 through October 1946, Albert Speer stood trial at Nuremberg as one of twenty-four major defendants. On October 1st, 1946, the International Military Tribunal convicted him of war crimes and crimes against humanity for his use of forced labor and sentenced him to twenty years. On July 18th, 1947, he was transferred to Spandau Prison in Berlin and registered as prisoner number five.

Margarete was forty-one. The youngest of her children was five. Ahead of her stretched two decades in which she would raise six children almost entirely alone, manage a household stripped of every privilege it had known, defend her husband’s reputation when she chose to, and stay silent when she chose not to. Of all the people Albert Speer left behind in 1945, no one’s life was reshaped more by his choices than hers.

The marriage had never been easy. Margarete had been born in Heidelberg on September 8th, 1905, the daughter of a master joiner who employed about fifty workers and served on the town council. She met Albert Speer in the summer of 1922, when both were teenagers. They married in Berlin on August 28th, 1928.

Seven years passed before Margarete was first invited to stay at her in-laws’ home. Her mother-in-law considered the Webers socially inferior. The seven-year wait for a welcome was the first lesson of the marriage in what she would and would not be allowed to expect. The Reich years made her the wife of one of the most powerful men in Germany and one of the most absent.

Between 1934 and 1942, she had six children. The regime awarded her a Cross of Honor for her fertility. Hitler gave the family a villa on the Obersalzberg in 1935. None of it changed the central fact of the marriage: Speer was almost never there.

Margarete herself put it bluntly in interviews decades later. For all practical purposes, the children did not have a father. He visited Berchtesgaden rarely, mostly when Hitler was in residence. He took his holidays with Eva Braun and his ministry team, not with his wife.

After 1945, she became the only parent the children had. When a reporter for Maclean’s interviewed her in early 1954, with Speer eight years into his sentence, she received him in the outhouse at Wolfsbrunnenweg, smartly dressed in a white blouse and gray skirt. She defended her husband to the foreign journalist with measured restraint. She refused to argue the case at Nuremberg.

She did not deny slave labor. She asked instead, quietly, whether eight years was not already enough. She had been raising six children alone since the surrender and would do so for twelve more years. She made one decision in those years that defined her relationship with her husband as much as anything else.

She refused to let any of the children visit him at Spandau. They would form a wrong impression, she told the reporter, of a father they had hardly known anyway. The children only began visiting in their teenage years on their own initiative. The release that should have been the end of the story was not.

Within a few years of his return, Albert Speer began an affair with a German woman thirty years his junior who was married to an Englishman living in London. Margarete and the children knew. She remained at the Heidelberg house. He died in the woman’s company in London on September 1st, 1981.

Margarete outlived him by six years and died at home on Christmas day, 1987. Albert Speer Jr. was born in Berlin on July 29th, 1934. By the time he turned three, his father was Hitler’s chief architect.

He grew up in the outhouse at Wolfsbrunnenweg with his mother and his siblings, and there the stutter arrived in 1945, when he was ten years old and his father was being arrested as a war criminal. It stayed with him into his thirties and only loosened its grip after a long bus trip across the United States in 1964, when he forced himself to speak to strangers in a language that was not his own. After finishing school, he made the decision that would define the rest of his life. He would become an architect—the same profession his father had used to design monuments for the Third Reich.

His grandfather, Albert Friedrich Speer, had practiced in Mannheim, and his great-grandfather before that. Albert Jr. took the longest possible route into the family trade. He worked first as a carpenter, beginning his apprenticeship while his father was still in Spandau, then enrolled in evening classes before entering architecture full-time at the Technical University of Munich in 1955.

In 1964, he won his first international prize and opened his own office. In 1977, he was appointed professor of urban planning at the University of Kaiserslautern. In 1984, he founded Büro Albert Speer & Partner in Frankfurt am Main, which grew into one of Germany’s most respected urban planning firms. The contrast with his father’s work was deliberate at every level.

Where Speer senior had designed at inhuman scale to make individuals feel small, Speer junior built around what he called human dimension: pedestrian-friendly streets, mid-rise density, ecological planning. He won few major commissions in Berlin, where his surname carried the heaviest weight, and built his reputation abroad. His firm was lead designer for the central axis of the 2008 Beijing Olympics, master planner for Expo 2000 in Hanover, a contributor to Shanghai International Automobile City, part of Munich’s bid for the 2018 Winter Olympics, and master planner for the Qatar 2022 FIFA World Cup. The irony was not lost on critics.

The son of Hitler’s architect repeatedly worked for governments that drew comparisons to the regime he had spent his career trying to escape. In interviews, he was direct about the arithmetic of his life. He told the Süddeutsche Zeitung that he had tried his entire life to separate himself from his father, and acknowledged that the man he had visited once a year in Spandau, the man who came out in 1966, was as foreign to him as one of his university professors. He told Reuters he had never considered changing his name.

He was the eldest son of that father and saw no reason to take another. The name had not helped him. Albert Speer Jr. died on September 15th, 2017, in Frankfurt am Main, aged eighty-three, from complications following surgery after a fall at home.

His firm continues to operate. In the summer of 1953, the United States refused her visa. Hilde Speer was seventeen, the second of the six children, and had been awarded an American Field Service Scholarship to study at Hastings High School on the Hudson, just outside New York City. The decision was reversed only after public protest, including offers of hospitality from American Jewish families.

She spent the year living with the family of Dr. Richard Day, a children’s doctor and psychologist, with the identity of her father kept officially secret in the locality. Her father’s name had become a question that opened with no and required others to argue for yes. Born on April 17th, 1936, Hilde was nine when the war ended—old enough to remember Hitler’s hand on her shoulder in photographs at the Berghof, young enough to be reshaped by what came after.

In Heidelberg, she attended a school originally founded by Elisabeth von Thadden, a resistance figure executed by the regime in 1944. One of her teachers there, Dora Lux, was a Jewish historian who had survived the war in Berlin. Lux became one of the formative influences on Hilde’s life. Hilde would later write a biography of her.

Lux shaped a question Hilde never stopped asking: What should the children of perpetrators do with what they had inherited? Politics came next. She joined the Greens during their breakthrough years in the 1980s and was elected to the Berlin House of Representatives, serving from 1985 to 1987 and again from 1989 to 1991. From 1989 to 1990, she served as the chamber’s vice president.

Her platform was consistent throughout: the peace movement, women’s rights, anti-racism, and a refusal to look away from what Germany had done. She married, raised children, became an education professor, and then in 1994 did something that defined her more than any office ever had. That year she inherited several paintings from her father’s collection. Investigation suggested they had probably not been looted from Jewish owners, but she could not bring herself to keep them.

She sold the paintings and with the proceeds founded the Zurückgeben Foundation—a German word meaning give back or return. The foundation funded grants to Jewish women working in the arts and sciences in Germany. By 2019, it had supported more than 130 projects with around 500,000 euros. The same year, the city of Berlin awarded her the Moses Mendelssohn Prize for her work promoting tolerance.

In 2019, she received the Obermayer German Jewish History Award. She gave a phrase to the Associated Press that captured a position none of her siblings ever quite reached. She had always felt fortunate, she said, because she had known what her father was and what he had done very early. Many men and women of her generation had no such answers about their own families.

She was still hosting refugees from Syria and Afghanistan in her Berlin home well into her eighties. In the 1990s, the photographer working on the Berlin exhibition Topographie des Terrors—a permanent installation on the site of former Gestapo headquarters, documenting the regime that had run it—was Albert Speer’s daughter. Margret Speer had been born on June 19th, 1938, named after her mother, and had spent the first six years of her life at Obersalzberg, a few hundred meters from the Berghof. She studied archaeology at Ruprecht-Karls-Universität, married the archaeologist Hans Nissen on April 14th, 1962, and lived in Baghdad from 1965 to 1967, then in Chicago, and finally in Berlin, where she taught herself photography and built a career in architectural work.

In 2004, she published her memoir, “Zinzi: Die Tochter Speer, Are You Speer’s Daughter? ,” through Deutsche Verlags-Anstalt in Munich. The title was the question she had been asked her entire adult life. The book described a father who was egocentric, distant, and far more invested in constructing the image he left behind than in being present for the children who carried his name.

She named his crimes plainly. She did not seek to exonerate him. Arnold Speer was born in 1940. His original name was Adolf, given in the year his father’s career reached its peak.

After the war, the family quietly renamed him Arnold. Public information about his life is limited. By various accounts, he became a community doctor and stayed out of public view. Two of the six made a choice that left even less behind.

Fritz, born in 1937, and Ernst, born in 1942, gave no interviews, wrote no memoirs, and made no public statements about their father. In a family where the surname guaranteed a question every time it was spoken, silence was its own kind of answer. Six children inherited one of the heaviest surnames of the twentieth century, and each made a different choice with it. One designed cities for the modern world.

One funded restitution for those their father’s regime had harmed. One photographed the regime’s archives and named what her father had done. Three said almost nothing at all. After their father’s death, they lived in different cities, worked in different professions, and never came together as a family.

Their mother had held the household together for thirty-six years. After 1987, even that connecting figure was gone.