The Call Stack

9 February 2021

The Call Stack

The call stack is the part of a program that nobody designs and everybody depends on: a contiguous region of memory and one register that points into it. On x86-64 that register is rsp; on ARM and AVR it is sp. Everything else about it – frame layout, calling conventions, how far it is allowed to grow – is convention layered on top of that single pointer.

The mechanism underneath is smaller than most people expect. call pushes the address of the next instruction onto the stack and jumps to the target. ret pops that address back into the program counter and jumps to it. That is the whole of it. A subroutine call is a store, a return is a load, and the nesting discipline falls out for free because the store and the load are last-in, first-out.

The stack grows down on x86, ARM and AVR: every push decrements the pointer. The reason is less architectural elegance than address-space economy. If the stack starts high and grows down while the heap starts low and grows up, the two share one contiguous region and meet in the middle, and neither has to know in advance how much the other will want. A machine with an upward-growing stack has to reserve a fixed gap between them at link time and wastes whatever the program does not use.

What is actually in a frame

A frame is the slice of stack belonging to one invocation. Its contents are decided by the calling convention, not by the language:

  • the return address, pushed by call;
  • the arguments that did not fit in registers;
  • the saved frame pointer, if the function keeps one;
  • the callee-saved registers the function intends to clobber;
  • local variables that could not stay in registers, and any temporaries spilled there;
  • padding, because x86-64 requires rsp to be 16-byte aligned at a call boundary.

How arguments arrive varies more than the rest of the layout. The 32-bit x86 cdecl convention pushes them right to left and leaves cleanup to the caller – which is exactly what makes variadic functions possible, since only the caller knows how many arguments it passed. stdcall, used by the Win32 API, gives cleanup to the callee and saves a few bytes per call site at the cost of that flexibility. The System V AMD64 convention passes the first six integer arguments in rdi, rsi, rdx, rcx, r8 and r9 and the first eight floating-point arguments in xmm0 through xmm7, spilling only the remainder to memory. ARM's AAPCS puts the first four arguments in r0r3 and, because ARM has a link register, keeps the return address in lr rather than on the stack – a leaf function there can return without touching memory at all.

The frame pointer, and why it disappears

Compilers traditionally keep a second register – rbp on x86-64 – pointing at the base of the current frame. Frames then form a linked list: from rbp you can walk to the caller's rbp, and so to the caller's caller, which is how a debugger produces a backtrace with no other information available.

At -O2 that register is usually reclaimed as a general-purpose register. On x86-64 there are only fifteen of them and one is a real cost, so the trade is made in favour of the optimiser. Backtraces then depend on DWARF call-frame information, a table the compiler emits describing how to recover the return address at any instruction address. It works well, but it needs the debug data to be present, and it makes a raw stack dump considerably harder to read. Embedded work often turns the frame pointer back on for exactly that reason: a crash on a device you cannot attach a debugger to is worth one register.

The red zone

One detail of System V AMD64 that surprises people writing signal handlers: the 128 bytes below rsp belong to the current function and may be written without adjusting the pointer at all. Leaf functions use this to avoid the two instructions a stack adjustment would cost. Signal handlers and interrupt entry code must not touch that region, because the interrupted function is entitled to it and will not expect it to have changed.

Overflow

The stack is finite and nothing in the instruction set enforces the bound. Recursion without a base case walks the pointer down until it leaves the region the operating system mapped – on a hosted system, usually into a guard page that faults cleanly; on a bare-metal microcontroller, into whatever else lives in RAM, silently. On a Cortex-M part the main stack and the process stack are separate and there is a stack limit register, which an RTOS can use to catch overflow at the next exception rather than at the next crash. On smaller parts there is nothing, and the failure mode is a corrupted variable somewhere else in memory, surfacing hours later.

The usual mitigation is a stack canary: the compiler places a known value between the locals and the saved return address and checks it on return. It catches the classic contiguous buffer overflow, and it does nothing at all about a non-adjacent write, an overwrite of a function pointer held further up, or a return-oriented attack that never needs to touch the canary. It is a useful speed bump, not a boundary.

Measuring it on a device

With no reliable runtime bound, the practical technique is a high-water mark. Fill the stack region with a known pattern – 0xA5 is the convention – run the firmware through its worst case, then scan upward from the bottom and find the deepest byte still holding the pattern. The distance from there to the top is what the program actually used.

The trap is that the answer is only true of the binary you measured. Debug and release builds have different frame sizes, an inlined function has no frame at all, and an interrupt taken at the worst possible moment adds a frame the main loop never produces. A stack that fits in testing and not in the field is not bad luck; it is a measurement taken over the wrong set of executions. The honest approach is to measure the deepest call chain with interrupts enabled at the optimisation level that ships, and then leave margin on top.

One structure, several names

The same object is called the call stack, the control stack and the run-time stack depending on which decade and which community is talking. Control stack is the oldest of the three and the most descriptive: the stack holds the control state of the program, the pending return addresses that define where execution resumes. Run-time distinguishes it from anything the compiler resolves statically. They are the same region of memory, and the differences between the names are entirely historical.

Tagged , , , ,


Testimonials

Dr. Mateja Verlič (Zemanta d.o.o., R&D)

We read a lot about people doing important things, however this time we – LOD2 partners – are the ones changing the history of Web by revolutionizing information use and reuse, contributing to semantic data standards and pushing the limits of almost every WWW-related aspect (sc… read

Dr. Jens Lehmann (Universität Leipzig, Research Group Leader)

The size and number of Semantic Web knowledge bases published as Linked Open Data has been growing tremendously over the past years. The LOD2 project will be a key factor for sustaining this momentum. More importantly, the quality of knowledge bases and the scalability of methods accessing… read

Jindřich Mynarz (University of Economics, Czech Republic)

Apart from being legally open, linked open data is an open technology. Due to its non-exclusive, non-proprietary, well-formalized, and standards-based nature, linked open data supports a wide spectrum of uses. It does not exclude any application from using it and thus it is open to be mixe… read

Dr. Mladen Stanojevic (Institute Mihajlo Pupin, Serbia)

The richness of any semantic model and the usability of represented data is dependent on the links between them. LOD2 is an important step in this direction that will enable an efficient exploration and processing of vast quantity of open data on the Web.

read

Mun Yong Yi (Korea Advanced Institute of Science and Technology (KAIST))

Data becomes more meaningful and powerful as they are linked and integrated. As I learn more about what LOD can do, I am convinced that it will not only change the future of the Internet but also the quality of human life. I am glad that I am participating in this project, which will defin… read

Orri Erling (OpenLink Software, Virtuoso Program Manager)

Value from information increasingly depends on integration. RDF is a great model for this. LOD2 will make RDF a cost competitive alternative in the database space, without compromising ad hoc flexibility and expressive power. read

Vojtech Svatek (University of Economics, Czech Republic)

As researcher in ontological engineering, I am excited to see LOD2 provide tools that make the generation of semantically structured data easy and thus widespread. I believe that ontological research is deemed to build upon the world views already expressed by means of simpl… read

Andreas Blumauer (Semantic Web Company, CEO)

15 years ago we all were excited when we published HTML for the first time and it didn't take a long time until all of us were "on the internet". Now we are starting to publish data on the web. Based on semantic web technologies professional data management will be possible in distributed … read

Kingsley Idehen (OpenLink Software, CEO)

Three years ago, OpenLink Software enthusiastically contributed the prowess of Virtuoso to the grassroots effort that lead to DBpedia and the Linked Open Data cloud that coalesced around it. Today, we are both honored and enthusiastic about Virtuoso's critical infrastructure role in this n… read

Bastiaan Deblieck (TenForce, Partner and Business Development Manager)

An internet of data opens up tremendous opportunities for our corporate and government customers. We intend to be on the forefront of this evolution. read

Christian Dirschl (Wolters Kluwer, Content Architect)

Linked (Open) Data will change the existing publishing paradigms! Creating high quality content for professional usage will remain an important factor in future publishing, but additional access points and new usage environments will equally define its success. read

Dr. Giovanni Tummarello (National University of Ireland, Galway, Research Unit Leader)

Semantic Markups on the Web could drive information reuse to enable scenarios and applications which we can now only dream of. The idea is extraordinarely compelling, but we know now it won't simply realize itself. The LOD2 project is now a great opportunity for inspired and coordinated re… read

Gregory Grefenstette (Exalead, Chief Science Officer)

Enterpise search is all about providing correct, complete, and appropriate information to the employee and decision maker. LOD2 promises to not only allow internal company information to be linked up to the growing amount of Open Data on the web, but to also provide the mechanisms for val… read

Hugh Williams (OpenLink Software)

It is exciting to see the LOD2 project finally kick off in its quest to take the Linked Open Data cloud to the next level of scalability, performance and integration for the exploitation of the Web as a viable platform for enterprise level data and information integration. read

Martin Kaltenböck (Semantic Web Company, CFO)

Linked (Open) Data technologies offer a new way of data integration for the enterprise! Smooth interoperability between internal data sets can reduce costs as well as the enrichment of these data sets by external data can support new market intelligence paradigms for a better decision making. read

Dr. Peter Boncz (Centrum Wiskunde & Informatica)

The publishing of ever more datasets by e.g. governments adds value for many key applications including business intelligence, which will drive the Linked Open Data (LOD) paradigm going forward. In the LOD2 project, CWI is working to increase the scalability and performance of querying int… read

Dr. Sören Auer (Universität Leipzig, LOD2 coordinator)

The Linked Data paradigm is a simple and efficient way for integration of heterogeneous information on the Web. Ultimately, we will, for example, be able to search for a new appartment and a close-by available spot in child care in one go. read

Tassilo Pellegrini (Semantic Web Company, R&D)

Semantic interoperability changes the technological and economic nature of metadata opening up exciting opportunities for value creation in various comercial and non-commercial areas. Linked Data is the blueprint for this new ecosystem and it will change the way we think about and use the web today. read

Wouter Dewanckel (TenForce, WP Project Leader)

It is an honor to take part in this challenging integration project to create solutions that can generate business value out of emerging technologies. read

News

IEEE publication of ?Virtuoso, a Hybrid RDBMS/Graph Column Store?

Apr 23, 2012 4:55:17 PM

My article, Virtuoso, a Hybrid RDBMS/Graph Column Store (PDF), can be found in Volume 35, Number 1, March 2012 (PDF) of the Bulletin of the IEEE Computer Society Technical Committee on Data Engineering (also known as the IEEE Data Engineering Bulletin). Abstract: We discuss applying column store techniques to both graph (RDF) and relational data for mixed workloads ranging from lookup to analytics in the context of the OpenLink Virtuoso DBMS. In so doing, we need to obtain the excellent m ...

News from the CKAN team 19 April 2012

Apr 19, 2012 4:36:28 PM

Here’s an update on what the CKAN team have been doing lately. It’s been a while since the last one and we’ve been busy, so there’s plenty to report. Features Adrià has implemented a great map view in Recline, CKAN’s built-in data viewer. If some structured data resource contains latitude and longitude information, this will enable it to be viewed on a map from within CKAN. A sneak preview is here (select the ‘map’ view). Ross has done some work on a ‘Related Stuff’ extensi ...

ICDE 2012 (post 6 of 6) - Science Data Panel

Apr 17, 2012 9:36:28 PM

Michael Stonebraker chaired a panel on the future of science data at ICDE 2012 last week. Other participants were Jeremy Kepner from MIT Lincoln Labs, Anastasia Ailamaki from EPFL, and Alex Szalay from Johns Hopkins University. This is the thrust of what was said, noted from memory. My comments follow after the synopsis. Jeremy Kepner: When Java was new we saw it as the coming thing and figured that in HPC we should find space for this. When MapReduce and Hadoop came along, we saw this as a se ...