Benoit Mandelbrot was a Polish-born mathematician most famous for his… (European Pressphoto Agency )

What do coastlines, clouds, cauliflower and the stock market have in common?

Mathematician Benoit Mandelbrot may not have conceived the question, but he provided an answer — one that was compelling in its originality and startling in its usefulness in fields as unrelated as geography, medicine, art and finance.

Mandelbrot, 85, who died of cancer Oct. 14 in Cambridge, Mass., was the father of fractals, a term he coined in 1975 to describe a new branch of geometry that seeks to make sense of irregular shapes and processes, from the infinite zigs and zags of a seacoast to erratic fluctuations on Wall Street.

His death was announced by Yale University, where he became a mathematics professor in 1987 after a long career at IBM in New York.

"He was a person who knew a great deal of mathematics and could see its impact in the world around him in a way no one else had," said Ralph Gomory, the former president of the Alfred P. Sloan Foundation who for many years headed IBM's research division.

Mandelbrot worked in obscurity for the first few decades of his career, persisting with mathematical explorations that struck other researchers as exceedingly esoteric if not incomprehensible. With the books "Fractals: Form, Chance and Dimension" (1977) and "The Fractal Geometry of Nature" (1982), he became the math world's equivalent of a rock star — the Carl Sagan of the numbers set — who drew standing-room audiences to lectures in United States and abroad.

His fractal theory entered popular culture, inspiring a song ("Mandelbrot Set" by Jonathan Coulton), novels (by Arthur C. Clarke and David Foster Wallace) and elegant, computer-generated art. It even enthralled a presidential candidate, Al Gore, who said during a 2000 campaign interview with the New York Times that he found fractals "an incredibly important way of looking at the world."

Classical Euclidean geometry describes flat surfaces, but Mandelbrot wondered about the nature of everything that wasn't flat, such as a cumulus cloud, a rock fragment or the craggy surface of the lungs. "Benoit realized that we are surrounded by objects best described as having not one or two or three dimensions but something in between," Gomory, also a mathematician, said in an interview this week.

Mandelbrot's first major insight stemmed from his attempt in the late 1960s to answer the question "How long is the coast of Britain?" He argued that the answer depends on the size of one's ruler. The more he zoomed in on the shoreline, the more indentations became visible. Classical geometry, conceived on the flat paradigm, could not handle this "roughness" or fractional dimension of non-linear shapes in the natural world, so Mandelbrot developed a new tool, which he later called fractal geometry. Instead of ignoring the disorderly twists and turns of the coast, mathematicians and geographers could now more accurately envision its scope and character.

"Why is geometry often described as 'cold' and 'dry'? One reason lies in its inability to describe the shape of a cloud, a mountain, a coastline, or a tree. Clouds are not spheres, mountains are not cones, coastlines are not circles, and bark is not smooth, nor does lightning travel in a straight line," he wrote in what would become the most quoted passage from "The Fractal Geometry of Nature."

He also found that shapes that seem random are actually highly ordered and self-repeating. The cauliflower, for instance, became one of his favorite illustrations of the fractal. "If you cut one of the florets of a cauliflower," he said earlier this year in a talk at the Technology Entertainment and Design conference, "you see the whole cauliflower but smaller."

The mathematical formula describing this phenomenon became known as the Mandelbrot Set. Programmed into a computer, the formula produces images of arresting beauty: infinitely varied amoebic shapes with intricate borders consisting of bumps and swirls that are actually embedded miniatures of the whole. Fractal art, popular on the Internet, T-shirts and posters, became icons of post-modern math.

Harvard mathematician David B. Mumford, speaking to the New York Times in 1985, said Mandelbrot's gift was his ability to "show people that they were making faulty assumptions simply because they didn't have the tools to look beyond them. They had missed a whole range of things. Mandelbrot's idea was that you could draw a thing, and that by drawing it you suddenly became aware of what was really going on. And the things he drew, I don't think that even he anticipated that they would turn out to be so strikingly beautiful."

In recent years, Mandelbrot gained attention for his application of fractals to Wall Street stock charts. In "The (Mis)behavior of Markets" (2005), written with Richard L. Hudson, he showed how investors underestimate the risks in the financial markets by giving more weight to day-to-day price movements and ignoring the effect of the precipitous swings.