Installing and Building the OpenOffice Prototype
Edward Peterlin, Dan Williams
9/30/02
Contents
Introduction
OpenOffice.org has recently reached a crucial milestone, the beta build for Darwin. The project has now entered the next major phase, migration to a Quartz implementation and Aquafication. This undertaking will probably take a significant amount of time and a large expenditure of resources. As such, it makes sense to take some time to explore some of the potential paths for execution so a year from now we don't find we made the wrong choice of strategies.
OpenOffice is an exploration of the strategy of executing an OS X native port on top of the existing Cocoa+QuickDraw VCL implementation. It is a complete hack job, is unstable, rails the processor, leaks memory, has drawing glithces, is mostly unmaintainable code, and does not integrate well with the potentially changing VCL of OpenOffice.org in future versions. It does, however, illustrate that the existing infrastructure can be pushed to create results that are fairly desirable.
Through this prototyping process we discovered several bugs and issues that make this approach undesirable:
- NSApplications on 10.2 seem to require running within a bundle in order to get keyboard focus. Since we couldn't run NO directly in a bundle, a wrapper executable similar to our installer was devised. It's not perfect, but it does at least allow for keyboard focus to be obtained by the application.
- Not using the AppKit [NSApp run] framework results in lots of hacks and code that may be unmaintainable for future AppKit revisions, not to mention railing the processor with a totally inefficient event loop.
- Threading with AppKit calls can get problematic. Hopefully 10.2 is happy with pthreads that have had autorelease pools allocated for them, but this is still a problem. Using NSThreads also proved problematic, although there is code in sal/osl/unx/thread_macosx.m for exploring usage of NSThreads.
- While creating windows on threads is functional, handling events on those threads in conjunction with the main event loop handling events leads to deadlocks and crashes. Our hack solution to this has been to, when possible, ensure dialogs run on the main thread.
- AppKit seems to want its event loop or nextEvent polling calls to all occur on the main thread or else bad things start happening. An attempt at fixing this has been introduced to vcl/aqua/source/app/VCLApplication.m to schedule events on the main thread to be communicated to alternative threads, but it is most likely non-functional.
- Using the NSQuickDrawView introduces a new problem to be solved: the port buffer needs to be explicitly flushed to draw to the screen. This was addressed by using a flushing thread per port and requesting flushes via a semapohre with collapsing of multiple requests occurring during a set time period (50 ms) into a single request. This method probalby has high overhead, but accomplishes its goal.
- QuickDraw does not work in threaded environments. All graphics calls are synchronized through use of a single global mutex. Additionally, QuickDraw has this nasty habit of segfaulting if you happen to draw outside of the boundaries of the view. This is an inefficient drawing mechanism that can only be addressed by moving to a saner thread safe drawing model, such as CoreFoundation or Cocoa native path drawing.
- The ATSUI support seems to work, but is slow. Some fonts also seem to crash ATSUI in the kerning pairs methods. Since we are doing explicit line layout ourselves, we may need to use ATSUI even if we move away from QuickDraw. ATSUI still seems to get some rendering incorrect from time to time, specifically on question marks and rs.
- Using the Appearance Manager for rendering can prove interesting at times with explicit erases necessary to draw shadows and the like.
- As we chose to retain the VCL event handling code for widgets, we only used native routines for control rendering and did not use native controls directly. This added another layer of cruft to the OutputDevice/SalGraphics combo. Ideally we should probably investigate using native controls with native event handling directly, but this is difficult in a mixed QuickDraw/Cocoa environment.
- We achieved an Aqua background by using an explicit bitmap stored in the resource files of the VCL and svtools modules. This works, but it is slow and suffers from misaligned backgrounds for controls within windows or controls within other controls. The alignment of the background could probably be fixed with some cleverness.
- Printing is not addressed.
- The ImpEditEngine exposed a problem either with drawing or with VirtualDevice usage. It simply produced a black rectangle on the screen when typing. This resulted in many things in the program becoming unusable. This was worked around by forcing the edit engine to not use the virtual device approach, but instead erase its destination area and draw directly into the window. This still exhibits problems in sdraw.
- The ordering of thread termination on exit seems to be a quite thorny issue and will more often then not result in crashes on exit. I believe this is due to the threads getting into a state where one therad is blocked on a condition while the main thread has attempted to release the primordial autorelease pool of the application. This may be solved if we were using NSThreads and the [NSApp run] methodology.
Requirements
- The fastest machine you can find. At least 300 megs of memory.
- MacOS X 10.2. OpenOffice will not build or run on earlier OS versions.
- December 2001 Development Tools for OS X.
- approx. 4 GB free HD space for a non-debug build. Source directory: 2.73 GB, setup directory: 104 MB, install directory: 224 MB. For a full debug build, roughly increase the space requirements by 2.5 times.
- A few cases of beer used to pass the time. On a TiBook 400 the build took about two continuous days.
Getting OpenOffice and External Sources
Once you have the requirements for building OpenOffice met by performing a small contract job or assuming a decent amount of credit card debt for a new hard drive, it's time to get the OpenOffice sources and other external sources. If you've built OpenOffice from previous sources and have applied other patches, you will need to start either with a source tarball, or with the CVS sources.
To download the CVS distribution:
- Open a new Terminal (/Applications/Utilities/Terminal)
- mkdir /neooffice
- cd /neooffice
- setenv CVSROOT :pserver:anoncvs@anoncvs.neooffice.org:/cvs
- cvs login
- Enter the password anoncvs
- cvs co OpenOffice
- go get some coffee, or a beer, since it'll take a while to get it all
After you get the CVS distribution you will need to take some steps to set up a build properly. OpenOffice is derived from OO638C_MacOSX and requires gcc2 to build. The first step is to therefore configure your build box to use gcc2:
- sudo gcc_select 2
Next we build the STLPort code. This is a compiler-agnostic STL implementation used with the OOo sources. To obtain the source:
- cd /neooffice
- tar xvf macosx_extras/STLport-4.0.tar
- cd STLport-4.0
- /neooffice/solenv/unxmacxp/bin/patch-STLport-4.0
- Enter /neooffice/STLport-4.0 as the directory when prompted
- cd /neooffice/STLport-4.0/src
- /usr/bin/make -f gcc-apple-macosx.mak
- go get some more coffee, or another beer, since it'll take a while to build
After getting stlport built you will need to get the dlcompat library for linking convienence. This is needed to properly build the patches for the Darwin port.
- Download the dlcompat package from the OSXGNU.org project.
- Uncompress it and double-click the .pkg file to install the library.
Important Note: If you've installed dlcompat from a different source location, you must be sure that it's properly linked to /usr/local/lib or else the OS X build will fail.
Building OpenOffice
Now that we've got it all patched up, it's time to execute the build!
- Open up a new Terminal.
- cd /neooffice/config_office
- ./configure --without-x --with-stlport4-home=/neooffice/stlport-4.0
- Hit Enter/Return at the "Please enter Java home directory" prompt.
- Enter "n" and hit return at the "Directory does not exist. Do you want to create it?" prompt.
- cd /neooffice
- source MacosxEnv.Set
- ./bootstrap
- source MacosxEnv.Set (this is to rescan the PATH to locate zipdep and other executables built during the bootstrap phase)
- dmake
- Begin consuming those cases of beer. You'll be waiting about a day.
Assembling the Setup Executable
After all of OpenOffice is built, the next step is to construct the appropriate directory so we can run setup and install the binaries we've just built. From what I know, it's not possible to run them straight out of the build directory.
- Open up a Terminal if you closed the other one in the past couple of days.
- mkdir /neooffice-setup
- cp /neooffice/instsetoo/unxmacxp.pro/01/normal/f* /neooffice-setup/
- cp /neooffice/instsetoo/unxmacxp.pro/01/normal/setup.ins /neooffice-setup/
- cp /neooffice/solver/638/unxmacxp.pro/lib/libstaticmxp.dylib /neooffice-setup/
- cd /neooffice-setup
- unzip f0\*
- cp -f /neooffice/setup2/unxmacxp.pro/bin/setup.app/Contents/MacOS/setup /neooffice-setup/setup.bin
- ln -s libcppu.dylib.3.0.0 libcppu.dylib.3
- ln -s libsal.dylib.3.0.0 libsal.dylib.3
- ln -s libstore.dylib.3.0.0 libstore.dylib.3
- ln -s libreg.dylib.3.0.0 libreg.dylib.3
Installing OpenOffice Binaries
Now setup has all the necessary libraries and files it needs to run. The setup process will exhibit several failures, but does properly register the recognized components if it's run correctly.
- Open a new Terminal. Do not re-use an existing terminal that has been previously used to build OpenOffice.
- mkdir /neooffice-install (this is where the final executables will live)
- cd /neooffice-setup/
- setenv DYLD_LIBRARY_PATH /neooffice-install/program:/neooffice-install/program/filter
- sh setup
- When prompted for intallation directory, specify /neooffice-install as the target directory. When you're running OpenOffice binaries from the terminal it isn't possible to use the keyboard. Click on the "Browse..." button to get a file dialog. Navigate to the neooffice-install folder in the dialog and click "OK".
- When prompted to "Please insert the disk...", click "OK" each time
- There will be errors that pop up for libraries and files containing the following: dbp, spa, psp, README, LICENSE. Hit the "Ignore" button in the error prompts to continue with the installation regardless.
- If you get dozens of "libfoo638mxp.dylib couldn't be registered", other than dbp, spa, and psp, then your DYLD_LIBRARY_PATH didn't match the installation directory. Delete ooo-install and run setup again
Once setup has exited, we need to move some files manually. Setup hasn't been fixed to work without a hitch on OS X:
- cd /neooffice-install/program/
- cp /neooffice-setup/libstaticmxp.dylib .
- cp -f /neooffice/desktop/unxmacxp.pro/bin/soffice.app/Contents/MacOS/soffice soffice.bin
- cp /neooffice/solver/638/unxmacxp.pro/bin/ooo63801.res resource/iso63801.res
What More Needs to be Done?
This patch is just a first step towards getting a fully functional build. There are a number of things which still need to be addressed before we can claim absolute victory.
- Integration with Mainline Sources: This patch is incredibly unstable in its current incarnatin, but we should begin migrating it to OO641/OOO_STABLE_1 sources. This patch contains several places where code may need to be ifdefed MACOSX out or commented out in the build files. Currently the patches may break other ports. If someone is working on two different ports, this may be the ideal project to start off with after the first build is done :)
- Integration and Evolution: We need to get this patch in a state safe for other ports, move to OO641 in preparation for OO1_STABLE, finish up all of the missing parts, etc. etc. etc. In other words, this patch represents the completion of the OO638C build step in our project roadmap and it's time to take up the ball and start heading for the endzone.
Building Test Programs
OpenOffice contains a number of test programs that can be used to exercise various parts of OpenOffice. These test programs are very useful for trying to find problems in the source code that may cause errors while running OpenOffice. The vcl test programs are incredibly useful for debugging problems when building Aqua graphics.
Once you have built the modules in the Abstraction and Infrastructure Layers, you are now ready to build the test programs for these layers. To build the test programs, execute the following commands:
> cd $SRC_ROOT/svtools/workben ; dmake ; deliver
> cd $SRC_ROOT/stoc/test/testsmgr_cpnt ; dmake ; deliver
> cd $SRC_ROOT/stoc/test ; dmake ; deliver
> cd $SRC_ROOT/bridges/test/java_uno ; dmake ; deliver
The commands listed above will build the following test programs. As these test programs are meant to provide a means for developers to debug the modules that are in the Abstraction and Infrastructure Layers, developers are highly encouraged to run these test programs on a platform such as Windows, Linux, or Solaris to determine what additional porting development effort is needed:
- Graphic System Layer Project test program - This test program contains code that exercises OpenOffice's VCL library. To run this test program, execute the following command:
> cd $SRC_ROOT/svtools/unxmacxp.pro/bin
$SRC_ROOT/svtools/unxmacxp.pro/bin> svdem
- UNO Development Kit Project test programs - These test programs contain code that exercises OpenOffice's UNO libraries. To run these test programs, execute any of the following commands:
> cd $SRC_ROOT/stoc/unxmacxp.pro/bin
$SRC_ROOT/stoc/unxmacxp.pro/bin> testconv
$SRC_ROOT/stoc/unxmacxp.pro/bin> testcorefl
$SRC_ROOT/stoc/unxmacxp.pro/bin> testintrosp
$SRC_ROOT/stoc/unxmacxp.pro/bin> testinvocation
$SRC_ROOT/stoc/unxmacxp.pro/bin> testloader
$SRC_ROOT/stoc/unxmacxp.pro/bin> testproxyfac
$SRC_ROOT/stoc/unxmacxp.pro/bin> testregistry
$SRC_ROOT/stoc/unxmacxp.pro/bin> testsmgr
> cd $SRC_ROOT/bridges/unxmacxp.pro/bin
$SRC_ROOT/bridges/unxmacxp.pro/bin> test ../misc/interfaces.rdb
Building Individual Projects
OpenOffice is organized in several projects. For example, the Word Processing Project. These in turn consist of several modules, organised in separate directories. The source contains approximately 90 modules.
You can build any project or module individually. Building modules individually should not be misunderstood as reducing OpenOffice to a special application, say, for instance, the spreadsheet application. The program will always consist in the entire office suite: text processor, spreadsheet, drawing application etc. Building individual modules comes in handy if you want to develop on a certain module or port a module that has not yet been ported for this plaform. Most modules will depend on other modules to be already built. In other words, all modules must build in a particular order. Accordingly, you must build all of the modules in the Abstraction and Infrastructure Layers before you trying building any projects or modules individually.
For more information on modules and on the sequence that they build in, and on the dependencies, see tools.openoffice.org/modules.html.
To build a project, you build each of its modules individually in their directory with the build
tool.
> cd $SRC_ROOT/(module-name)
> $SRC_ROOT/(module-name)> build
> $SRC_ROOT/(module-name)> deliver
Files called
makefile.rc
in each directory with further subdirectories iterate through all directories of the module and executes
dmake
in each of them (just like the top-level
makefile.rc
does when building the entire office suite). The last or second to last directory is usually
module-name/util
which is responsible for linking one or more shared libraries.
Building a Project with Debug Information
To rebuild a complete project with debug information, remove all object files by removing the unxmacxp.pro
directory. Then run dmake
with the debug option set to true:
> cd $SRC_ROOT/(module-name)
> $SRC_ROOT/(module-name)> rm -Rf unxmacxp.pro
> $SRC_ROOT/(module-name)> build debug=true
> $SRC_ROOT/(module-name)> deliver
Re-running Setup to Reinstall OpenOffice
After you've built a module with debug symbols, chances are you'll want to reinstall OpenOffice in order to run OpenOffice with debugging information. The first thing you'll need to do is remove setup, the install directory, and some config files. If you installed to the paths suggested above, this can be accomplished with the following set of commands:
- rm -rf /ooo-install
- rm -rf /ooo-setup
- rm ~/.sversionrc
After getting rid of the setup directory, install directory, and the .sversionrc file we need to recreate the setup executable. First we need to reconstruct the zip files in instsetoo:
- cd /openoffice
- source MacosxEnv.Set
- cd instsetoo
- build
Now you can resume the instructions in Assembling the Setup Executable and complete with the installation. You can then get debug symbols by gdb sh and then issuing r /path/to/soffice
Deleting a Build Directory
When you're deleting a directory containing a checkout of the OpenOffice sources, you should rm -rf it from a terminal. If you are trying to delete your checkout using the Finder, you'll run into a fun little feature of the OS X Finder that doesn't allow you to delete folders named desktop. OpenOffice has a folder with this name in it, and this folder will need to be deleted from the terminal.
Moving or Renaming a Build Directory
The build process for OpenOffice utilizes hardcoded paths to STLPort and other components of the source tree. If you rename your OpenOffice build directory or if you move it to a new location, you should delete all binaries and start from scratch by runnint config_office/configure again. To erase all your object files:
- cd $SRCROOT
- find . -name unxmacxp.pro -print0 | xargs -n 1 -0 rm -rf
If you do not perform this step, you will most likely be unable to build or link out of your build directory any longer.