With its startup consumer bank Marcus, Goldman Sachs (GS) wants to disrupt banking the way Amazon (AMZN) disrupted retail and and Apple (APPL) upended music.

Specifically, the 150-year old bank’s goal is to transform "the distribution and consumption experience of financial services," according to Harit Talwar, the head of global consumer business for Goldman Sachs.

"Think about it — a mom wants to buy diapers. The diapers she buys at Walgreens or the diapers she buys at Amazon are the same diapers,” Talwar explained Yahoo Finance last week, on the sidelines of the Fortune Brainstorm Finance event in Montauk, New York.

“But, with Amazon she's convinced of [the] value, it is easy, it is simple, it's transparent, it's personal, it's repeatable, and she does it from her family room,” he said. “There is no reason why you need to walk to a branch to do banking.”

A startup within Goldman

Just three years ago, the storied Wall Street firm debuted Marcus, Goldman’s online consumer lending and savings arm. It was named after the bank's founder, Marcus Goldman, and described as "a startup within a 150-year old firm."

Adam Dell, Marcus’ head of product, explained that they want to help customers understand that they are paying their current bank $150 to $200 in fees each year and they're getting very little in interest on their savings account.

For its savings arm, Marcus offers a 2.25% annual percentage yield (APY) and charges no fees. Customers can open an account for as little as $1. Customers can also get access to fixed-rate, no-fee loans of up to $40,000.

To date, Marcus now has around 4 million customers, and it's given out north of $4.7 billion in consumer loans, and accumulated close to $50 billion in deposits.

More recently, Marcus partnered with Apple for the upcoming Apple Card, a digital and physical credit card that's expected to launch this summer.

"We are growing at a quite rapid rate. I think that's a reflection of the value proposition of our business," said Dell, who raised eyebrows last week when he declared that big banks were “screwed.”

For Marcus, it’s only the beginning.

"You will see us move into other categories of finance whether it be investing, retirement, insurance, all of the things that make up a person's financial life are fair game," Dell said.

A noted venture capitalist and entrepreneur, Dell joined Goldman as a partner last year after selling Clarity Money to the bank.

He analogized Marcus to Amazon Web Services (AWS). Basically, Amazon invested heavily in offering its cloud infrastructure to other businesses, and it's become a massive money maker for the e-commerce giant.