MAYA FILE TRANSLATOR

Florian LOITSCH

Trainer: Diane LINGRAND

Co-Trainer: Benjamin PRUCHA

Summary:

During this project a file-translator for Maya (otherwise known as exporter) is going to be developed. This file-translator exports a given Maya-scene into an enhanced proprietary format. Wootsoft's current format should be used as basis, but improvements are to be made.

Furthermore a small viewer needs to be implemented to test the correct export and import of the models.

Context:

Wootsoft is a small enterprise, situated in Austria. Two years ago it started the development of “Iritor Online” (a Massive Multiplayer Online Role Playing Game). From the beginning on, Wootsoft employed several artists, who created a great amount of models in Maya (a major 3D content creation tool). Already at that time the export-process wasn't optimal: An artist (or in rare cases a programmer) exported the model into an X-file (the DirectX-specific file-format). Then a small utility was run over these files, and extracted the Iritor Online relevant information. During this process the X-file was split into up to three separate files, to share information between different models.

Shortly before the beginning of this project an internal decision had been made to port the game onto the Linux platform, and to rewrite the 3D-engine under OpenGL. While it was (and is) still possible to export models using the old process, it is clearly not the ideal way to go: X-files are Microsoft specific, and are not supported under Linux. It is hence impossible to convert an X-file to an IOM-file (Wootsoft's proprietary file format) under Linux. Furthermore, the switch from DirectX to OpenGL induced a switch from left-handed to right-handed coordinate-system. Therefore an adapted converter is needed. Wootsoft finally decided to assign this project to me to write a native Maya-exporter, that would export into a new IOM-format.

Requirements

The project is split into two major parts:

• the Maya-exporter (in the Maya-language a “file-translator”) and C++ importer.

• a platform-independent fast small model-viewer, that allows to inspect the exported data.

Maya-exporter

Maya (by Alias Wavefront) is a well-known 3D content creation tool, that exists on several platforms: MS Windows, Linux and MacOSX. By providing an open C++ API and a powerful scripting language (MEL), Alias Wavefront invites programmers to write plug-ins for Maya.

The C++ API interfaces directly with the Maya-internals, and MEL gives a way to write platform-independent UI-dialogs. Using Maya's shared libraries, it is also possible to write stand-alone executables, that work on Maya-scenes. This allows small tools, without the need of installing the full Maya suite.

As Wootsoft isn't satisfied with their current proprietary file-format a new one should be developed. While the initial idea of sharing data between different models seems to be a good idea, it was actually never used, and made things more complicated. Consequently, the new file-format should contain all necessary information for a model in one file. If possible the file-format should provide the possibility to add eventually textures to it, so a model-file would be self-contained.

Once the file-format has stabilized, an Maya-Plug-in has to be written. This plug-in should interface with a small MEL-script, that allows the user to change several options. Those options aren't yet defined, but it should be easy to add/remove new choices for Wootsoft-employees.

Maya provides a way of writing stand-alone programs (using their API), and it would be nice to have a small tool, that extracts the model out of a Maya-file without starting Maya. While this isn't necessary, it would be nice for batch-jobs, etc.

In the next paragraph I'm finally presenting the requirements for the exporter itself and explain the exported data. Iritor Online works with two different model-types, which need to be exported differently:

• Static models

  Static models are clearly the easiest to export: the exported data consists of one big mesh, where each vertex of that mesh contains the following elements:

• A position in world-space.

• A normal. If a vertex has two different normals, the vertex is exported twice.

• UV-coordinates (aka ST-coordinates under OpenGL). Again, if a vertex contains two different sets of UV-coordinates, the vertex is exported twice.

• Animated models
  Animated models are way more complex to export: three components are extracted out of the Maya for them:

• the bone-structure: Instead of animating all vertices, a simpler bone-structure is created, onto which the vertexes are attached. Artists only animate those bones, instead of the whole set of vertexes.

• a skin: this is an enhanced version of the previous mesh (for the static model). As every vertex is affected by up to four bones, additionally to the already exported elements, four bone-indices and four bone-influences are exported. The bone-index points to the influencing bone, and the influence determines the force this bone is affecting the vertex.

• the animation-data (using key-frames): The animation-data consists of simple functions. For key-frames (that the artists carefully choose during their model-creation), a transformation-matrix is exported. Using these matrices the bones can be animated. As the skin is attached to the bones, a complete animation is exported this way.

In addition to the listed items, an additional “shadow-mesh” needs to be exported for most objects. Contrary to the previously cited meshes a shadow-mesh doesn't contain UV-coordinates. Furthermore a shadow-mesh needs to be “closed”. This means, that, even after moving a vertex no gaps are allowed. While the exporter can't modify the mesh to satisfy these conditions, it can enforce them, and warn the artists in case of problems. Often helpful advices (like the position of the problem-vertex) are appreciated.

A Maya-scene stores other useful information, that could be used at the game-side. For instance the texture-filenames are contained in every Maya-file. Optionally the exporter could include as much as possible of these information. As it is not yet clear, what data could be extracted out of Maya, the file-format needs to be kept very flexible.

Model-Viewer

Until now Wootsoft uses the game as model-viewer. Every time a model needs to be inspected the model is included into the game, and then viewed in the game. In the beginning this wasn't a problem, as loading time was short, and it wasn't very difficult to add a new object to the game.

Over time the load-time increased, and viewing models has become cumbersome. As the C++ import functions need to be tested anyways, a small viewer should be created.

Completed Work

The completed work can be split into 3 major parts: the time spent on finding a new file-format. The work done on the exporter itself, and finally the Viewer.

.IO42

The new file-format (using the extension IO42, for Iritor Online and the magic number 42), isn't actually very complicated, and still very open for changes. But this was a goal during conception.

Instead of defining a whole new file-format, that nobody would ever be able to read without a proper importer, I decided to use a zip-library to combine several smaller files. So instead of having (up to) three different files for one model, as has been the case with the old file-format, there will only be one.

This gives several advantages over the old format:

• One self-contained file for every model.

• Expandable. There is really no difficulty in adding new data to a model. The data could be added during the export-process (like texture-names), manually (like TODO-files, comments) or automatically during a batch-process using other ZIP-tools.

• The contained files are simple and hence easy to read. For instance, a texture can still be viewed without problems, using a simple Image-viewer.

• Several models could be grouped into one bigger model. The specifications of the file-format is not yet fixed, but one could easily join several models, by either creating sub-folders, or just including (recursively) the model-files into a new zip-file.

• The data is compressed, and hence uses less space, which is especially nice for the non-compressed TarGA-files, that are usually used as textures.

Illustration 1 contents of jorgalosk.io42

A simple example of such a file is shown in Illustration 1, where jorgalosk.io42 contains a mesh-file, animation-data, and a texture.

Until now it hasn't been decided what mesh-format will be used, but if a standard-format (like Maya's .obj) is going to be used, most 3D viewers would be able to view the static mesh (or the skin for animated models).

To implement this new file-format the exporter needs to be able to save information into a ZIP-file. Fortunately I found (after some research), a BSD-style ZIP-library, that suits perfectly our needs: zlib. As this library is heavily used in major GNU-programs they are believed to be stable, and seem to be a good choice.

Exporter

Although Maya is a great and well-know 3D program, it was until recently very expensive, and only reserved to big companies. Furthermore these companies usually avoided publishing their exporters (and knowledge in generally) of Maya. This makes sample-source code very rare, and difficult to find. While the Maya-API is well documented, the documentation rarely exceeds a JavaDoc-style: every class is documented, but big concepts, and “standard ways of doing” aren't explained anywhere. The provided code-samples are next to useless for my exporter, and due to the richness of the Maya API there is always more than one way of doing something (although not always all ways are correct).

Most of the time has hence been spent on researching more information, and on trying different ways of exporting.

Until now I have found ways of exporting static meshes including their normals, and UV-sets. The result can be seen in the Viewer-screenshots below (Illustration 2 and 3 on page 8). On the bright side: this exporter works also as stand-alone program, and doesn't need Maya to be running (which is very convenient for batch-processes).

Exporting animation on the other hand requires a thorough understanding of Maya internals. The open-source DirectX x-file exporter contains about 7500 lines, where one of the more important functions itself measures 2000 lines (which makes reverse-engineering very difficult).

Therefore most of the time has been (and is) spent on reverse-engineering, and researching.

Model Viewer:

After I had successfully exported my first model, Benjamin Prucha sent me their graphics-engine, so I could start implementing the viewer. Although I first encountered some starting-difficulties (the new code wasn't yet very stable and easy to compile), it's now in a working state. I added the code to load my exported models, and the viewer is now capable of showing static meshes and mapping hard-coded textures onto the model. This reflects the current state of the exporter, and normally the viewer should from now on advance at the same pace, as the exporter, as every new feature of the exporter is immediately tested in the viewer. See Illustration 2 and 3 for some screenshots.

Planning

Past

Future

Personal Conclusion

Knowledge is Power. Computer Science is one of those domains, were this proverb applies perfectly. The code for the current exporter (admitted still not feature-complete), is not longer than several hundreds of lines. It's amazing how much time has been spent on finding those lines. Wootsoft (and especially Benjamin Prucha) helped me quite often, but sometimes I was just stuck for several hours, trying to find a simple line to be able to continue.

On the other hand, developing the new file-format, and discussing it with Wootsoft's employees was quite fun. Creativity was much more important during this process.

References

David A. D. Gould “Complete Maya Programming”, Morgan Kaufmann, 2003

http://www.alias.com/eng/products-services/maya/index.shtml (Alias Wavefront's Maya)

http://www.ewertb.com/maya/api/ (Maya API How-To page)