Memory Hierarchy

The closer the memory is to the processor, the faster and often wider the connection.  Registers and first level caches are running at processor speed, and sometimes transfer multiple words per cycle.  Then there can be more caching between these immediate processor buffers and the system memory, which is almost always DRAM where each memory cell is a single gate and a single capacitor.

DRAM has been popular as system memory for a long time, and the current generation can pack 16Gbit on a single chip.  An SoC can have this main memory on or off chip, but it will almost always include the memory controller.  This controller not only generates the proper timings for the DRAM read and write from the processor and cache or custom registers and devices, but it keeps the memory refreshed.  The one drawback of DRAM is that the charges on the capacitors leak out and periodically you have to visit each row to read and re-write the data to keep it fresh.

We can have other kinds of memory connected directly to the processor.  Flash memory generally needs to be read out into system memory because of its interface requirements, but there are a number of ROM technologies that can be used to store fixed data and programming.  Many designs favor the flexibility of being able to re-program these memories to fix bugs and make improvements, but the more hard-programmed, the more dense and lower power.

The other big drawback of DRAM, as you may have guessed from its need for refreshing, is that it needs to have power to remember anything.  Turn it off, and poof, memory gone.  Since the days of magnetic cores, we haven't had a non-volitile main memory technology.  The current solid state contender is Flash memory, which is pretty good but not perfect.  You can't just keep writing over again and again, it wears out.

A the edge memory transitions from being a local component to being access to independent systems over a network. Even if the flash storage is on the chip, because it is accessed just like other storage devices, we can lump it abstractly with storage devices and networks where persistent databases and files are stored.  Some flash drives pretend that they are mechanical media like magnetic disks, only solid state.  They replace laptop drives directly by matching the physical pattern of the original disk drives.  A flash component can be integrated on the SoC and/or plugged into a Micro SD slot that most SoC boards have.

Historically, storage was big drive cabinets that eventually became the laptop scale 2.5 inch drives at ever greater densities.  In a data center there are often dedicated high speed networks connecting trays of processors to trays of drives that can be quickly re-configured with software. For an SoC, they typically has two network connections, USB for devices from disk drives and optical drives to keyboards and microphones, and Ethernet and/or Wireless interfaces to connect to the Internet.  Because a lot of SoCs are built for Cell phones, some of them will have GSM right on the chip, but if the one you are using doesn't, there are modules that do.