mac dump_syms: Support .dSYMs > 4GB (partially)

Even 64-bit Mach-O (MH_MAGIC_64 = 0xfeedfacf) is not a fully 64-bit file
format. File offsets in sections are stored in 32-bit fields, with
Mach-O writers typically truncating offsets too large to fit to just
their low 32 bits. When a section begins at a file offset >= 4GB,
dump_syms would produce an error such as:

Google Chrome Framework.dSYM/Contents/Resources/DWARF/Google Chrome Framework: the section '__apple_names' in segment '__DWARF' claims its contents lie outside the segment's contents

As a workaround, this implements the strategy I first described in
https://crbug.com/940823#c22.

Segment file offsets are stored in 64-bit fields. Because segments
contain sections and must load contiguously, it’s possible to infer a
section’s actual offset by computing its load address relative to its
containing segment’s load address, and treating this as an offset into
the containing segment’s file offset. For safety, this is only done for
64-bit segments (LC_SEGMENT_64) where the 32-bit section offset stored
in the Mach-O file is equal to the low (truncated) 32 bits of the
section offset recomputed per the above strategy.

Beware that this does not provide full “large file” support for 64-bit
Mach-O files. There are other file offsets within Mach-O files aside
from section file offsets that are stored in 32-bit fields even in the
64-bit format, including offsets to symbol table data (LC_SYMTAB and
LC_DYSYMTAB). No attempt is made to recover correct file offsets for
such data because, at present, such data is always stored by dsymutil
near the beginning of .dSYM files, within the first 4GB. If it becomes
necessary to address these other offsets, it should be possible to
recover these offsets by reference to the __LINKEDIT segment that
normally contains them, provided that __LINKEDIT doesn’t span more than
4GB, according to the strategy discussed at the bottom of
https://crbug.com/940823#c22.

Although this is sufficient to allow dump_syms to interpret Chromium
.dSYM files that exceed 4GB, be warned that these Mach-O files are still
technically malformed, and most other tools that consume Mach-O files
will continue to have difficulties interpreting these large files.

As further warning, note that should any individual DWARF section exceed
4GB, internal section offsets will be truncated irrecoverably, unless
and until the toolchain implements support for DWARF64.
https://bugs.llvm.org/show_bug.cgi?id=14969

With this change, dump_syms is able to correctly recover file offsets
from and continue processing a .dSYM file with length 4530593528
(4321MB), whose largest section (__DWARF,__debug_info = .debug_info) has
size 0x8d64c0b8 (2262MB), and which contains four sections (starting
with __DWARF,__apple_names) beginning at file offsets >= 4GB.

Bug: chromium:940823, chromium:946404
Change-Id: I23f5f3b07773fa2f010204d5bb53b6fb1d4926f7
Reviewed-on: https://chromium-review.googlesource.com/c/breakpad/breakpad/+/1541830
Reviewed-by: Robert Sesek <rsesek@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
2 files changed
tree: ee28a1fc9b43534693d47e037a3eef426d5997b4
  1. .github/
  2. android/
  3. autotools/
  4. docs/
  5. m4/
  6. scripts/
  7. src/
  8. .gitignore
  9. .travis.yml
  10. aclocal.m4
  11. appveyor.yml
  12. AUTHORS
  13. breakpad-client.pc.in
  14. breakpad.pc.in
  15. ChangeLog
  16. codereview.settings
  17. configure
  18. configure.ac
  19. default.xml
  20. DEPS
  21. INSTALL
  22. LICENSE
  23. Makefile.am
  24. Makefile.in
  25. NEWS
  26. README.ANDROID
  27. README.md
README.md

Breakpad

Breakpad is a set of client and server components which implement a crash-reporting system.

Getting started (from master)

  1. First, download depot_tools and ensure that they’re in your PATH.

  2. Create a new directory for checking out the source code (it must be named breakpad).

    mkdir breakpad && cd breakpad
    
  3. Run the fetch tool from depot_tools to download all the source repos.

    fetch breakpad
    cd src
    
  4. Build the source.

    ./configure && make
    

    You can also cd to another directory and run configure from there to build outside the source tree.

    This will build the processor tools (src/processor/minidump_stackwalk, src/processor/minidump_dump, etc), and when building on Linux it will also build the client libraries and some tools (src/tools/linux/dump_syms/dump_syms, src/tools/linux/md2core/minidump-2-core, etc).

  5. Optionally, run tests.

    make check
    
  6. Optionally, install the built libraries

    make install
    

If you need to reconfigure your build be sure to run make distclean first.

To update an existing checkout to a newer revision, you can git pull as usual, but then you should run gclient sync to ensure that the dependent repos are up-to-date.

To request change review

  1. Follow the steps above to get the source and build it.

  2. Make changes. Build and test your changes. For core code like processor use methods above. For linux/mac/windows, there are test targets in each project file.

  3. Commit your changes to your local repo and upload them to the server. http://dev.chromium.org/developers/contributing-code e.g. git commit ... && git cl upload ... You will be prompted for credential and a description.

  4. At https://chromium-review.googlesource.com/ you'll find your issue listed; click on it, then “Add reviewer”, and enter in the code reviewer. Depending on your settings, you may not see an email, but the reviewer has been notified with google-breakpad-dev@googlegroups.com always CC’d.