Assembly HOWTO aka *Free* 32-bit x86 Assembly FAQ aka Linux x86 Assembly HOWTO Francois-Rene Rideau rideau@ens.fr v0.3c, 15 Jun 1996 HOWTO program in x86 assembly using only *FREE* programming tools. Keywords: assembly, assembler, free, macroprocessor, preprocessor, asm, inline asm, 32-bit, x86, i386, gas, as86, nasm LEGAL BLURP: Copyright (C) 1996 Francois-Rene Rideau. You can freely distribute this document, provided no modification is done to the text, but annotations that are clearly indicated as such. You can freely ask me to distribute the document otherwise. The Linux Documentation Project maintainers are free to do anything with this document, that all other LDP documents simultaneously allow, which they can understand optimistically as for a fix-point meaning. IMPORTANT NOTE: This is still a *VERY PRELIMINARY* version for this document. You (hey, that's *you* I'm talking to, so please listen!) are especially invited to ask questions, to answer to questions, to correct given answers, to add new FAQ answers, to give pointers to other software, to insult the current maintainer (me), and TO TAKE OVER THE MAINTENANCE OF THE FAQ in his place (mine), because I have other things to do... For any of these, please contact me mailto:rideau@ens.fr Perhaps we can convince Raymond Moon to add a section to his FAQ for comp.lang.asm.x86... ? 1. INTRODUCTION. This document aims at answering frequently asked questions of people who program or want to program 32-bit x86 assembly using *free* assemblers, particularly under the Linux operating system. It may also point to other documents about non-free, non-x86, or non-32-bit assemblers, though it is not its primary goal. Because the main interest of assembly programming is to build to write the guts of operating systems, languages, and games, where a C compiler fails to provide the needed expressivity (performance is more and more seldom an issue), we stress on development of such software. 1.1 How to use this document This document contains answers to some frequently asked questions. At many places, Universal Resource Locators (URL) are given for some software or documentation repository. Please see that the most useful repositories are mirrored, and that by accessing a nearer mirror site, you relieve the whole Internet from unneeded network traffic, while saving your own precious time. Particularly, there are large repositories all over the world, that mirror other popular repositories. You should learn and note what are those places near you (networkwise). Sometimes, the list of mirrors is listed in a file, or in a login message. Please heed the advice. Else, you should ask archie about the software you're looking for... The most recent version for this documents sits in http://www.eleves.ens.fr:8080/home/rideau/Assembly but what's in Linux HOWTO repositories *should* be fairly up to date, too (I can't know): ftp://sunsite.unc.edu/pub/linux/docs/HOWTO/ (?) 1.2 Other related documents * If you don't know what *free* software is, please do read *carefully* the GNU General Public License, which is used in a lot of free software, and a model for most; it generally comes in a file named "COPYING", with a library version in a file named "COPYING.LIB". Litterature from the FSF (free software foundation) might help you, too. * Particularly, the interesting kind of free software comes with sources that you can consult and correct, or sometimes even borrow from. Read your particular license carefully, and do comply to it. * There is a FAQ for comp.lang.asm.x86 that answers generic questions about x86 assembly programming, and questions about some commercial assemblers in a 16-bit DOS environment. Some of it apply to free 32-bit asm programming, so you may want to read this FAQ... http://www2.dgsys.com/~raymoon/faq/asmfaq.zip * FAQs and docs exist about programming on your favorite platform, whichever it is, that you should consult for platform-specific issues not directly related to programming in assembler. 2. ASSEMBLERS. 2.1 GCC Inline Assembly The well-known GNU C/C++ Compiler (GCC), an optimizing 32-bit compiler at the heart of the GNU project, supports the x86 architecture quite well, and includes the ability to insert assembly code in C programs, in such a way that register allocation can be either specified or left to GCC. GCC works on most available platforms, notably Linux, *BSD, VSTa, OS/2, *DOS, Win*, etc. 2.1.1 Where to find GCC The original GCC site is ftp://prep.ai.mit.edu/pub/gnu/ together with all the released application software from the GNU project. However, there exists a lot of mirrors. However, sources adapted to your favorite OS, and binaries precompiled for it, should be found at your usual FTP sites. For GCC under Linux, see around http://www.linux.org.uk/ For most popular DOS port of GCC is named DJGPP, and can be found in directories of such name in FTP sites. See: http://www.delorie.com/djgpp/ There is also a port of GCC to OS/2 named EMX, that also works under DOS; see around: http://www.leo.org/pub/comp/os/os2/gnu/emx+gcc/ http://warp.eecs.berkeley.edu/os2/software/shareware/emx.html 2.1.2 Where to find docs for GCC Inline Asm The documentation of GCC includes documentation files in texinfo format, that you can convert to tex, compile (with tex), and print, convert to interactive emacs .info format and browse, convert (with the right tools) to whatever you like, or just read as is. The .info files are generally found on any good installation for GCC. The right section to look for is: C Extensions::Extended Asm:: Section Invoking GCC::Submodel Options::i386 Options:: might help too. Particularly, it gives the i386 specific constraint names for registers: abcdSDB correspond to %eax, %ebx, %ecx, %edx, %esi, %edi, %ebp respectively (no letter for %esp). A URL for this document and section, as converted in HTML format, is http://www.cygnus.com/doc/usegcc_89.html#SEC92 The DJGPP Games resource (not only for game hackers) has this page specifically about assembly: http://www.rt66.com/~brennan/djgpp/djgpp_asm.html Finally, there is a web page called, "DJGPP Quick ASM Programming Guide", that covers URLs to FAQs, AT&T x86 ASM Syntax, Some inline ASM information, and converting .obj/.lib files: http://remus.rutgers.edu/~avly/djasm.html GCC depends on GAS for assembling, and follow its syntax (see below); do mind that inline asm needs percent characters to be quoted so they be passed to GAS. See the section about GAS below. Find *lots* of useful examples in the linux/include/asm-i386/ subdirectory of the sources for the free Linux OS. 2.1.3 How should I invoke GCC for it to properly inline my assembly code ? Be sure to invoke GCC with the "-O" flag, to enable optimizations and inline assembly. If you don't, your code may compile, but not run properly!!! More generally, good compile flags for GCC on the x86 platform are gcc -O2 -fomit-frame-pointer -m386 -O2 is the good optimization level. Optimizing besides it yields code that is a lot larger, but only a bit faster; such overoptimizationn might be useful for tight loops only (if any), which you may be doing in assembly anyway; if you need that, do it just for the few routines that need it. -fomit-frame-pointer allows generated code to skip the stupid frame pointer maintenance, which makes code smaller and faster, and frees a register for further optimizations. It precludes the easy use of debugging tools (gdb), but when you use these, you just don't care about size and speed anymore anyway. -m386 yields more compact code, without any measurable slowdown, (note that small code also means less disk I/O and faster execution) but perhaps on the above-mentioned tight loops. To optimize even more, option -mregparm=2 and/or corresponding function attribute might help, but might pose lots of problems when linking to foreign code... Note that you can add make these flags the default by editing file /usr/lib/gcc-lib/i486-linux/2.7.2/specs or wherever that is on your system. 2.2 GAS GAS is the GNU Assembler, that GCC relies upon, with 2.2.1 Where to find it Find it at the same place where you found GCC, in a package named binutils. 2.2.2 What is this AT&T syntax Because GAS was invented to support a 32-bit unix compiler, it uses standard "AT&T" syntax, which resembles a lot the syntax for standard 680x0 assemblers. This syntax is no worse, no better than the "Intel" syntax. It's just different. When you get used to it, you find it much more regular than the Intel syntax, though a bit boring. A program exists to help you convert programs from TASM syntax to AT&T syntax. See ftp://x2ftp.oulu.fi/pub/msdos/programming/convert/ta2asv08.zip A file gas.doc or as.doc (still around the same place as you found GAS, if not in the GAS source package itself) describes the syntax. One place for it is in FTP directory ftp://sunsite.unc.edu/pub/linux/GCC/ ftp://sunsite.doc.ic.ac.uk/packages/linux/sunsite.unc-mirror/GCC/ (?) Again, the sources for the Linux OS come in as good examples; see under linux/arch/i386, the following files: kernel/entry.S, kernel/head.S, boot/compressed/head.S, mathemu/*.S 2.2.3 Limited 16-bit mode GAS is a 32-bit assembler, meant to support a 32-bit compiler. It currently has only limited support for 16-bit mode, which consists in prepending the 32-bit prefixes to instructions, so you write 32-bit code that runs in 16-bit mode on a 32 bit CPU. In both modes, it supports 16-bit register usage, but what is unsupported is 16-bit addressing. Use the directive "code16" and "code32" to switch between modes. Note that an inline assembly statement asm("code16\n") will allow 32-bit GCC to produce code that'll run in real mode! Feel free to add full 16-bit support if you think you need it. 2.3 GASP GASP is the GAS Preprocessor. It adds macros and some nice syntax to GAS. 2.3.1 Where to find GASP I don't know exactly. See at the GNU repository (prep.ai.mit.edu & mirrors). Perhaps together with GAS in the binutils package ? 2.3.2 How it works I have no idea, but it comes with its own texinfo documentation, so just print them, or browse the .info files... Looks like a regular macro-assembler to me. 2.4 AS86 AS86 is a 80x86 assembler, both 16-bit and 32-bit, part of Bruce Evans' C Compiler (BCC). It has mostly Intel-syntax, though it differs slightly as for addressing modes. 2.4.1 Where to get AS86 A completely outdated version of AS86 is distributed by HJLu just to compile the Linux kernel, in a package named bin86 (current version 0.3), available in any Linux GCC repository. But I advise no one to use it for anything else but compiling Linux. This version supports only a hacked minix object file format, and has a few bugs in 32-bit mode, so you better keep it only for compiling Linux. The most recent versions are published together with the FreeBSD distribution. I got it from ftp://ftp.ibp.fr/pub/FreeBSD/packages-2.1/development/bcc-95.3.12.tgz But your mirror may vary, and the version may evolve. Among other things, it supports Linux GNU a.out format, so you can link you code to Linux programs, and/or use the usual tools from the GNU binutil package to manipulate your data. This version can co-exist without any harm with the previous one (see question 2.4.4 below). BCC from 12 march 1995 and earlier version has a misfeature that makes all segment pushing/popping 16-bit, which is quite annoying when programming in 32-bit mode. A patch is published in the Tunes project http://www.eleves.ens.fr:8080/home/rideau/Tunes/ subpage files/tunes.0.0.0.25.src.tgz in unpacked subdirectory LLL/i386/ The patch should also be in available directly from http://www.eleves.ens.fr:8080/home/rideau/files/as86.bcc.patch.gz Bruce Evans accepted this patch, so if there is a more recent version of bcc somewhere someday, the patch should have been included... Note for DOS users (?): * To compile it under DOS you might have to define POSIX_HEADERS_MISSING. * For bcc/as, if you're not using DJGPP, you will have to rename a variable named "far" in function mcall() in file mops.c, because some DOS compilers think "far" is a reserved keyword. Note that you need link with typeconf.obj from the bcc/ld directory... * For bcc/ld, you'll need have a copy of a.out.h and ar.h; DJGPP has them, but other C compilers may require that you steal them from any GCC (under DOS, Linux, VSTa, etc). * For bcc/ld, you need define the BSD_A_OUT macro for all files, and edit writebin.c so it define STANDARD_GNU_A_OUT and includes your copy of linux a.out.h with a usable DOS name. * Linux a.out in turns includes asm/a.out.h, so you must manage to get it included, too. 16-bit compilers require that you edit the asm/a.out.h to modify a 24-bit bitfield into an equivalently sized set of bitfields lesser than 24 bits (ld doesn't use that field). * I didn't try cc1, but it should be quite doable to compile it, too, if you like it; however, you'll have to rewrite part of the bcc frontend, or use cc1 directly, because it relies on the fork()/exec()/wait() treble to launch cc1, as, and ld when compiling. * precompiled DOS versions can be found on http://www.eleves.ens.fr:8080/home/rideau/files/asld86.zip If you would recompile them with free compilers and send me, I'd appreciate a lot. 2.4.2 Where to find docs The docs are what is included in the bcc package. Man pages are also available somewhere on the FreeBSD site. When in doubt, the sources themselves are often a good docs: it's not very well commented, but the programming style is clear. You /might/ try to see how as86 is used in Tunes 0.0.0.25... 2.4.3 How to invoke the assembler ? Here's the GNU Makefile entry for using bcc to transform .s asm into GNU a.out .o object and .l listing: %.o %.l: %.s bcc -3 -G -c -A-d -A-l -A$*.l -o $*.o $< Remove the "%.l", "-A-l", and "-A$*.l", if you don't want any listing. If you want something else than GNU a.out, you can see the docs of bcc about the other supported formats, and/or use the objcopy utility from the GNU binutils package. 2.4.3 What if I can't compile Linux anymore with this new version ? Linus is buried alive in mail, and my patch for compiling Linux with a Linux a.out as86 didn't make it to him (!). Now, this shouldn't matter: just keep your as86 from the bin86 package in /usr/bin, and put the good as86 as /usr/local/libexec/i386/bcc/as where it should be. You never need explicitly call this "good" as86, because bcc does everything right, including conversion to Linux a.out, when invoked with the right options; so assemble files exclusively with bcc as a frontend, not directly with as86. 2.5 OTHER ASSEMBLERS These are other, non-regular, options, in case the previous didn't satisfy you (why ?), that I don't recommend in the usual (?) case, but that could be useful if the assembler is part of what you're designing (i.e. an OS or development environment). 2.5.1 Win32Forth assembler Win32Forth is a *free* 32-bit FORTH system that successfully runs under Win32s, Win95, Win/NT. It includes a free 32-bit assembler (either prefix or postfix syntax) integrated to the assembler. Macro processing is done by the full power of the reflective FORTH language, but the only supported input and output contexts is Win32For itself. Find it at ftp://ftp.forth.org/pub/Forth/win32for/ 2.5.2 NASM The Netwide Assembler project tries to produce yet another assembler, written in C, that would be modular enough to eventually support all known syntaxes and object formats. Current version runs just fine, but only some very simple syntax, and plain binary output; No integrated macroprocessing. Surely NASM evolves too fast for this HOWTO to be up to date; still don't expect it to have all the required features currently, unless you're ready to help implement it. http://www.dcs.warwick.ac.uk/~jules/nasm1.html 2.5.3 Tunes The Tunes OS project is developping its own assembler as an extension to the Scheme language, as part of its development process. It doesn't run at all yet, though help is welcome. The assembler manipulates symbolic syntax trees, so it could equally serve as the basis for a assembly syntax translator, a disassembler, a common assembler/compiler back-end, etc. Also, the full power of a real languae, Scheme, make it unchallenged as for macroprocessing/metaprograming http://www.eleves.ens.fr:8080/home/rideau/Tunes/ 2.5.4 Non-free and/or Non-32bit x86 assemblers. You may find more about them, together with the basics of x86 assembly programming, in Raymond Moon's FAQ for comp.lang.asm.x86 http://www2.dgsys.com/~raymoon/faq/asmfaq.zip 3. METAPROGRAMMING/MACROPROCESSING Assembly programming is a bore, but for critical parts of programs. You should use the appropriate tool for the right task, so don't choose assembly when it's not fit; C, OCAML, perl, Scheme, might be a better choice for most of your programming. However, there are cases when these tools do not give a fine enough control on the machine, and assembly is useful or needed. In those case, you'll appreciate a system of macroprocessing and metaprogramming that'll allow recurring patterns to be defined once, and reused multiple times, by automatically inline expansion, which allows safer programming, easier modification, etc. A "plain" assembler is often not enough, even when one is doing only small routines to link with C. 3.1 What's integrated into the above 3.1.1 GCC GCC allows (and requires) you to specify register constraints in your "inline assembly" code, so the optimizer always know about it; thus, inline assembly code is really made of patterns, not forcibly exact code. Then, you can make put your assembly into CPP macros, so anyone can use it in as any C function/macro. and inline C functions Inline functions resemble macros very much, but are sometimes cleaner to use. Beware that in those cases, code will be duplicated, so only local labels (of "1:" style) should be defined in that asm code. However, a macro would allow the name for a non local defined label to be passed. Also, note that some bug in your code or in GCC may appear when inlining functions with asm code where the register constraints weren't declared properly and/or confuse GCC. Lastly, the C language itself may be considered as a good abstraction to assembly programming, which relieves you from most of the trouble of assembling. Beware that some optimizations that involve passing arguments to functions through registers may make those functions unsuitable to be called from assembly in the standard way, least you give them the attribute asmlinkage. See the linux kernel sources for examples. 3.1.2 GAS GAS has absolutely NO macro capability included. However, GCC and passes .S files through CPP before to feed them to GAS. .s files are the generated ones, and passed directly to GAS. Again and again, see Linux sources for examples. 3.1.3 GASP It adds all the usual macroassembly tricks to GAS. See its texinfo docs. 3.1.4 AS86 It has some simple macro support, but I couldn't find docs. Now the sources are very straightforward, so if you're interested, you should understand them easily. If you need more than the basics, you should use an external filter (see below in section 3.2) 3.1.5 OTHER ASSEMBLERS Win32FORTH: CODE and END-CODE are macros that do not switch from interpretation mode, so you have access to the full power of FORTH words, immediate or not, while assembling. NASM: no macro support yet, see about external filters below. TUNES: it doesn't work yet, but the Scheme language is a real high-level language that allows arbitrary meta-programming. 3.2 EXTERNAL FILTERS Whatever is the macro support from your assembler, or whatever language you use (even C !), if the language is not expressive enough to you, you can have files passed through an external filter with a Makefile rule like that: %.s: %.S other_dependencies $(FILTER) $(FILTER_OPTIONS) < $< > $@ 3.2.1 CPP CPP is truely not very expressive, but it's enough for easy things, it's standard, and called transparently by GCC. As an example of its limitations, you can't declare objects so that destructors are automatically called at the end of the declaring block, you can't co-declared data and the code to process it, etc. CPP came with your C compiler. If you could make it without one, don't bother fetching any (though I wonder how you could). GCC (see above) is a free C compiler you could have fetched. 3.2.2 M4 M4 gives you the full power of macroprocessing, with a Turing equivalent language, recursion, regular expressions, etc. You can do with it everything that CPP cannot. See macro4th/This4th from ftp://ftp.forth.org/pub/Forth/ in Reviewed/ ANS/ (?), or the Tunes 0.0.0.25 sources as examples of advanced macroprogramming using m4. However, its fucked up quoting semantics force you to use explicit continuation-passing tail-recursive macro style if you want to do *advanced* macro programming (which is remindful of TeX -- BTW, has anyone tried to use TeX as a macroprocessor for anything else than typesetting ?). This is NOT worse than CPP that does not allow quoting and recursion anyway. The right version of m4 to get is GNU m4 1.4 (or later if exists), which has the most features and the least bugs or limitations of all. 3.2.3 Macroprocessing with yer own filter You can write your own simple macro-expansion filter with the usual tools: perl, awk, sed, etc. That's quick to do, and you control everything. But of course, any power in macroprocessing must be earned the hard way. 3.2.4 Metaprogramming Instead of using an external filter that expands macros, one way to do things is to write programs that write part or all of other programs. For instance, you could use a program outputing source code to generate sine/cosine/whatever lookup tables, to extract a source-form representation of a binary file, to compile your bitmaps into fast display routines, to extract documentation, initialization/finalization code, description tables, as well as normal code from the same source files, to have customized assembly code, generated from a perl/shell/scheme script that does arbitrary processing, (particularly useful when some kind of data must be mirrored at into many cross-referencing tables and code chunks). etc. Think about it ! 4. CALLING CONVENTIONS 4.1 Linux 4.1.1 Linking to GCC That's the preferred way. 32-bit arguments are pushed on stack in reverse order (hence popped in the right order) above the 32-bit near return address. %ebp,%esi,%edi,%ebx are preserved, %eax holds the result, or %edx:%eax for 64-bit results. FP stack: I'm not sure, but I think it's result in st(0), whole stack callee-save. GCC has options to modify the calling conventions by reserving registers, having arguments in registers, not assuming the FPU, etc. Check the i386 info pages. Beware that you must put declare asmlinkage attribute for a function you want GCC to provide these standard conventions to (I don't know what it does with modified calling conventions). 4.1.2 ELF vs a.out problems Some C compilers prepend an underscore before every symbol, while others do not. Particularly, Linux a.out GCC does such prepending, while Linux ELF GCC does not. See how the Linux source tree manages that itself (linux/include/linux/linkage.h). You can override the C->asm renaming by inserting statements like void foo asm("bar") (void); to be sure that the C function foo will be called really bar in assembly. Note that the utility objcopy, from the binutils package, should allow you to transform your a.out objects into ELF objects, and perhaps the contrary too, in some cases. 4.1.3 Direct Linux syscalls This is NOT recommended, because it may change, it's not portable, AND it precludes libc fixes and extensions, like, for instance zlibc (that does on-the-fly transparent decompression of gzip-compressed files). The standard, recommended way to call Linux system services is to go through the libc. Now, if you don't want to link to the libc, you should see how linux-eforth-1.0c.tgz does it ftp://ftp.forth.org/pub/Forth/Linux/ The sources for Linux come in handy, too, particularly the asm/unistd.h header file, that describes how to do system calls... Basically, you issue an int $0x80, with the __NR_syscallname number (from asm/unistd.h) in %eax, and parameters (up to five) in %ebx, %ecx, %edx, %esi, %edi respectively. Result is returned in %eax, with a negative result being an error whose opposite is what libc would put in errno. The user-stack is not touched, so you needn't have a valid one when doing a syscall. 4.2 DOS Most DOS extenders come with some interface to DOS services. Read their docs about that, but often, they just simulate int 0x21 and such, so you do "as if" you were in real mode (I doubt they have stubs to have things work with 32-bit operands by calling 16-bit DOS services as needed). Docs about DPMI and such can be found on ftp://x2ftp.oulu.fi/pub/msdos/programming/ DJGPP comes with its own (limited) libc replacement, too. It is possible to cross-compile from Linux to DOS, but the existing patches were written for a.out GCC. Perhaps recent ELF GCCs need no patch, perhaps they need new patches; I just dunno. 4.3 YOUR OWN OS [That's what many asm programmers talk about] 4.3.1 Boot loader code & getting into 32-bit mode 4.3.2 The basics about protection 4.3.3 Handling Interrupts 4.3.4 V86/R86 mode for using 16-bit system services. 4.3.5 Where to find info about it all. [Please add pointers to other documents to this section] The main source for information is sources of existing OSes. Lots of pointers lie in the following WWW page: http://www.eleves.ens.fr:8080/home/rideau/Tunes/Review/OSes.html 5. TODO * fill incomplete sections * add more pointers to software... * add simple examples from real life to illustrate the syntax, power, and limitations of each proposed solution.