31 Aug

Extracting text from PSD files

Typically, we need to translate hundreds of strings when localizing our games. Most strings are text messages which the game loads from some database or simple text file. However, we often have to handle localization of several PNG images, such as the one below.

AWESOME! message in DragonScales

Such PNG images are exported from PSD files which must obviously contain at least one Text Layer. To speed up the localization process we have a little Photoshop script which opens the PSD files and extracts all the text we have to translate. PSD files are grouped in directories corresponding to the tileset they belong to. For instance, this would be a typical directory structure for the DragonScales games:

images/
    board/
        awesome.psd, great.psd, ...
    cards
        youwin.psd, sorry.psd, ...
    ending
        congratulations.psd, ...
    levelselect
        clickhere.psd, ...
    mainmenu
        welcome.psd, ...
    etc

A simplified but functional version of the script we use is this:

#target photoshop

var target = "/C/projects/ds/images";
var toLocalize = new Array();
var totalProcessed = 0;
var warningsFiles = new Array();

function log(msg) {
	$.writeln(msg);
}

function processPSDFolder(dir) {
	var files = dir.getFiles("*.psd");
    log("===============================================");
    log(dir + " -> "+ files.length);
    log("===============================================");
	for (var i = 0; i < files.length; i++) {
    	var doc = app.open(files[i]);
    	log("  file: " + files[i]);
        totalProcessed++;
        if ( doc.artLayers.length == 0 ) {
            log("   -> WARNING: ZERO TEXT LAYERS? THEY SHOULD NOT BE IN GROUPS.");
            warningsFiles.push(files[i]);
        }
    	for (var j = 0; j < doc.artLayers.length; j++) {
        	var lyr = doc.artLayers[j];
        	if (lyr.kind == LayerKind.TEXT) {
            	var lyr = doc.artLayers[j];
            	log("   ->" + lyr.textItem.contents);
            	toLocalize.push(lyr.textItem.contents);
        	}
     	}
    	doc.close(SaveOptions.DONOTSAVECHANGES);
	}
}

function saveStrings() {
	var out = new File(target + "/strings.txt");
	out.open("w");
	for (var i = 0; i < toLocalize.length; i++) {
		var str = toLocalize[i];
		out.writeln(str);
	}
 	out.close();
}

var root = Folder(target).getFiles();
for ( var i = 0; i < root.length; i++ ) {
	var fileFoldObj = root[i];
	if ( fileFoldObj instanceof File ) {         
        // Discard files at this level
	} else {
         processPSDFolder( Folder(fileFoldObj) );
	}
}
saveStrings();
log("Total PSDs processed: " + totalProcessed);
log("Warnings: " + warningsFiles.length);
for ( var i = 0; i < warningsFiles.length; i++ ) {
	log(" " + warningsFiles[i]);
}

Observations:

  • target is the path to your directory structure holding the PSD files.
  • The strings to be translated will we written to file strings.txt under your target directory.
  • This script looks for text layers on the top level of the PSD. It can be easily extended to inspect layers in groups, though.
  • We use warnings to be notified about files not containing Text Layers. These might be files requiring special exporting and extra formatting, and therefore we’ll have to handle such files exceptionally.
24 Feb

Comments about warning “Deprecated Carbon Component Manager” on OS X El Capitan

Last November, after updating to OS X “El Capitan”, we started seeing a strange warning message when running DragonScales 2:

“WARNING: 140: This application, or a library it uses, is using the deprecated Carbon Component Manager for hosting Audio Units. Support for this will be removed in a future release. Also, this makes the host incompatible with version 3 audio units. Please transition to the API’s in AudioComponent.h.”

DragonScales 2 was built with LibGDX, and before updating to “El Capitan” it did run with no problems, warnings, etc. After some research, we were informed that this was caused by an OpenAL-Soft issue (which has already been fixed.) Specifically, Apple is deprecating some libraries, e.g., the Carbon Component Manager, and the OpenAL-Soft library was referencing such deprecated Carbon Component. When a game referencing these deprecated libraries is executed, newer OS X (e.g., El Capitan) shows the above warning. However, as told, this OpenAL-Soft issue was already solved on last November: the library is updated and ready for prime time.

A build of OpenAL-Soft is part of the LWJGL natives bundled with LibGDX. If you don’t use an updated build of OpenAL-Soft you’ll keep receiving the deprecation warning. As we commented on this thread, a quick fix is downloading the latest LWJGL3 build, grab the native libopenal.dylib and drop it over the OpenAL native bundled with LibGDX.

A caveat, though, about using the OpenAL-Soft included in LWJGL3 (at time of writing.) Some testers of DragonScales 2 for OS X reported a nasty crash. Here’s part of the crash report:

Dyld Error Message:
Symbol not found: ___sincosf_stret
Referenced from: /var/folders/*/openal.dylib
Expected in: /usr/lib/libSystem.B.dylib

Clearly, our OpenAL library was referencing a function not provided by OS X. It turns out that our testers were using an older OS X version (10.7, I think) whereas the libopenal.dylib bundled with LWJGL3 was targeting OS X 10.9. In fact, output of otool -l libopenal.1.17.0.dylib includes these lines:

cmd LC_VERSION_MIN_MACOSX
cmdsize 16
version 10.9
sdk 10.9

As our publishers require support for OS X >= 10.7, we had to compile our own libopenal.dylib. We set OS X 10.6 as deployment target and 10.11 (El Capitan) as the root SDK, by using these CMake variables:

-DCMAKE_OSX_DEPLOYMENT_TARGET=10.6 -DCMAKE_OSX_SYSROOT=macosx10.11.

By the way, here you can find OS X SDKs if you need them.

Checking our build with otool, yields:

Load command 8
cmd LC_VERSION_MIN_MACOSX
cmdsize 16
version 10.6
sdk 10.11

This version of the library passed all our tests successfully.

27 Aug

Resizing multiple image files with ImageMagick

For the first DragonScales post I had several PNGs with a 1920×1200 resolution. Nevertheless, the ideal width for images in this blog is about 600 pixels:

DragonScales Logo (Black background)

DragonScales Logo (Black background)

This is a frequent issue when publishing images here. Thereby I recur to ImageMagick in order to resize sets of images. ImageMagick’s command line tools are excellent and indispensable tools for our work. For instance, on Windows command prompt, it’s enough to type:

forfiles /M *.png /C “cmd /c convert -resize 31% @file resized_@file”

That way all of the images will have resized versions, at 31% their original size, yielding a 600-pixels width approximately. Very useful. And ImageMagick is a great piece of software.

19 Aug

A weekend boxed inside a dylib

I’ve dedicated the entire weekend to heterogeneous yet related tasks. First, I started off by trying to build NagiQ, our first published game, on OS X Mavericks. By “build” I mean to produce a binary complying with the requirements of the Mac App Store. You know what I mean: well-formed directory hierarchies inside the .app, proper icons and .plists, code signing, etc. For the record, NagiQ is a word game created with Ren’Py, a visual novel engine which is, in turn, built with Python. At the time of release a lot of folks shared their thought regarding our election of Ren’Py to create our word game: it was, to say the least, an unorthodox choice. However, almost 3 years after its initial release, NagiQ is still running fairly well on Windows, Mac and Linux, thanks to the wonderful capabilities of Ren’Py for multi-platform deployment.

Building NagiQ on Mavericks is easy, it amounts to just a single click on a Ren’Py option[1]. However, turning the generated .app into a binary suitable for the Mac App Store has proven to be a daunting task. You have to organize the directory structure of the Python Framework distributed with the game. You have to circumvent the writing of Python .o files in the .app directory, a big no-no for sandboxed apps. You have to retrieve the proper directory to save user and game data (~/Library/Application Support/NagiQ is not allowed). Etcetera. Right on the middle of such etcetera lies the requirement of communication with a few dynamic libraries needed by NagiQ to satisfy several functional demands.

As you surely know, dynamic libraries = dylibs on Mac. Taking into account that Ren’Py is a citizen of the Python world, we use the ctypes library to communicate with our dylibs from inside the game. An important lesson I learned during this weekend is that you have to be very careful when dealing with dylibs and ctypes. First, you have to verify that your dylib and the Python version you’re running are compatible. Is your dylib a 32 or a 64-bits binary? Your Python instance must be apt to properly load and call functions of your dylib, or you will spend a lot of time trying to sort out segmentation faults.

Typing python on a terminal of my Mac launches the 64-bit version of the interpreter by default. If you want to execute the 32-bit version, run this in your shell before launching python:

export VERSIONER_PYTHON_PREFER_32_BIT=Yes

After you have launched the proper version of Python, you can import ctypes to load and communicate with your dylib. However, don’t take this communication lightly. Pay special attention to the type of the arguments and return values of your functions. For instance, if you’re invoking a function of your dylib which requires a char* value, then you have to wrap your argument in the type c_char_p. Let’s see another example. Suppose you want to get the proper location to save the data of your game. Of course, as a good programmer, you don’t want to hardcode such path. Instead, you’ll be asking the operating system for it, the right way. Let’s create a tiny, demonstrative Objective-C library (demolib.m) for this:

#import <Foundation/Foundation.h>
const char* findAppDir()
{
    NSArray *paths = NSSearchPathForDirectoriesInDomains(
        NSApplicationSupportDirectory,
        NSUserDomainMask, YES);
    NSString *basePath = [paths objectAtIndex:0];
    const char *convertedPath = [basePath UTF8String];

    return convertedPath;
}

Compile with:

clang -dynamiclib -framework Foundation demolib.m -o demolib.dylib

Then you can call your function from Python:

Python 2.7.5 (default, Mar  9 2014, 22:15:05)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> lib = ctypes.cdll.LoadLibrary("demolib.dylib")
>>> findAppDir = lib.findAppDir
>>> findAppDir.restype = ctypes.c_char_p
>>> findAppDir()
'/Users/yourusername/Library/Application Support'
>>>

This little function will prove useful later, when I resume the building of NagiQ. However, the weekend is already over and I have yet to implement several adjustments for DragonScales, our next game soon to be released. The delight of working with dylibs, sandboxes, etc., will surely be the subject of future happy weekends.

  1. [1]This single-step build has grown to be the Ren’Py feature I love the most.
29 Apr

Lampson’s Hints for Computer System Design

Although it’s a bit dated in parts (mostly the examples), the article Hints for Computer System Design by Butler W. Lampson is still a valuable resource for today’s designers and engineers. It’s a nice reading, with plenty of timeless advice. For instance, regarding the issue of how to handle all the cases, the article says:

Handle normal and worst cases separately as a rule, because the requirements for the two are quite different:

  • The normal case must be fast.
  • The worst case must make some progress.

In most systems it is all right to schedule unfairly and give no service to some of the processes, or even to deadlock the entire system, as long as this event is detected automatically and doesn’t happen too often. The usual recovery is by crashing some processes, or even the entire system. At first this sounds terrible, but one crash a week is usually a cheap price to pay for 20% better performance. Of course the system must have decent error recovery (an application of the end-to-end principle…), but that is required in any case, since there are so many other possible causes of a crash.

The full article can be found at Microsoft Research.

14 Mar

Programming: A Constructive Art

In the preface of “Algorithms + Data Structures = Programs” (Prentice-Hall Series in Automatic Computation, 1976; page xv), Nicklaus Wirth stated (boldface mine):

Programming is a constructive art. How can a constructive, inventive activity be taught? One method is to crystallize elementary composition principles out of many cases and exhibit them in a systematic manner. But programming is a field of vast variety often involving complex intellectual activities. The belief that it could ever be condensed into a sort of pure “recipe teaching” is mistaken. What remains in our arsenal of teaching methods is the careful selection and presentation of master examples. Naturally, we should not believe that every person is capable of gaining equally much from the study of examples. It is the characteristic of this approach that much is left to the student, to his diligence and intuition. This is particularly true of the relatively involved and long example programs. Their inclusion in this book is not accidental. Longer programs are the prevalent case in practice, and they are much more suitable for exhibiting that elusive but essential ingredient called style and orderly structure. They are also meant to serve as exercises in the art of program reading, which too often is neglected in favor of program writing. This is a primary motivation behind the inclusion of larger programs as examples in their entirety. The reader is led through a gradual development of the program; he is given various snapshots in the evolution of a program, whereby this development becomes manifest as a stepwise refinement of the details. I consider it essential that programs are shown in final form with sufficient attention to details, for in programming, the devil hides in the details. Although the mere presentation of an algorithm’s principle and its mathematical analysis may be stimulating and challenging to the academic mind, it seems dishonest to the engineering practitioner. I have therefore strictly adhered to the rule of presenting the final programs in a language in which they can actually be run on a computer.

Several decades later, I think Wirth’s observations about teaching and learning to program are still thoroughly valid. In fact, even today, the art of program reading is still neglected in favor of program writing. Careful reading and analysis of others’ code is a great resource for learning and improving our programming skills. And, of course, great minds are timeless.

28 May

Measuring a programmer’s productivity

Measuring a programmer’s productivity is a difficult chore mainly because analyzing the final output of a programmer’s activity (software, usually) can be a very challenging task. Furthermore, what are our guidelines for identifying good software? Speed of execution (or of development), needed space, conformance to requirements, being readable, battery consumption…? All of these and even more? During my college years I remember taking a course on Assembly language. Our teacher used a simple scoring method: the best program (and thereby the program receiving the maximum grade) would be the one with the smallest count of lines of code, and of course the program would also have to generate a correct output. Well, the lines-of-code scheme is a simple, direct measuring system, but it’s rarely effective. I was reminded of this college course by this text I read today at computerhistory.org:

When the Lisa team was pushing to finalize their software in 1982, project managers started requiring programmers to submit weekly forms reporting on the number of lines of code they had written. Bill Atkinson thought that was silly. For the week in which he had rewritten QuickDraw’s region calculation routines to be six times faster and 2000 lines shorter, he put “-2000″ on the form. After a few more weeks the managers stopped asking him to fill out the form, and he gladly complied.

Needless to say, another proof of Bill Atkinson‘s genius.

09 Mar

In regard to the discussion on the JVM being a viable platform for mobile games

The author of LibGDX has just published an excellent and reflective post: THE JVM – A VIABLE PLATFORM FOR (MOBILE) GAMES?. A thought-provoking reading. Our humble answer to this question is yes. But things could be better, much better. For instance, as we all know, the approach to port your LibGDX game to iOS, albeit effective and a victory of cleverness, is notoriously contrived. It feels like we are working with fine porcelain which could break anytime, anywhere. Even for the desktop, Java game development and distribution can pose unexpected challenges. For instance, bundling your jar for distribution on Windows can face surprising hurdles:

  1. We cannot provide further details because of NDAs, but there are some DRM wrappers which don’t interact well with launch4j outputs.
  2. Bundling has to take special care to handle the “Missing msvcr71.dll” problem (Java 6. For Java7 your problem will be msvcr100.dll). Take a look at this.

On Mac OS X, things might get even harder. You’ll have to use the AppBundler, work through several configuration files, and even compile your own JRE.

Regarding mobile deployment, well, if you’re using LibGDX, Android is your most natural mobile target. iOS is possible too, but it’s not that natural mostly because Java is not a first-class citizen there. Blackberry devices, which allow for Java development, are not possible at the moment with LibGDX because BB has no JNI support. For Bada you’d use C++. For Firefox OS you’d use HTML 5. Tizen is based on HTML5 (may support Java too). Play Station Mobile requires C#. Windows Phone 8 allows C++, C# and Visual Basic, I think.

That said, we’re not big Java fans. Honestly, we’d rather work on C++, or Python, or C#, or even PROLOG, and we really don’t care that much for the other languages that target the JVM. We’d rather work on Vim with a bunch of makefiles. However, after completing our first game, NagiQ, a word game for desktop, we wanted to be able to deploy our next titles on mobile devices, specifically, Android devices. As Google insists on Java as your first language of choice, we followed the Java/LibGDX route. The great part here was LibGDX. Java is a good language, but what we really enjoyed was using LibGDX: its design is very, very good. What we want from a framework/SDK/library is good abstraction layers and cross-platform capabilities, and LibGDX excels at offering both. Furthermore, you can go lower level if needed, and you can browse and alter the source code if needed. That’s wonderful. We are much less interested on tools. Our dream: LibGDX in C++, coding with Vim, having a set of makefiles for automagically creating versions of our games for desktop and several mobile targets (Android, iOS, BB, PSM, etc). A dream. Just a dream.

Our feeling is that the JVM is getting behind. C# is becoming the preferred managed solution out there on mobile devices. For the time being, our plan is to stick to LibGDX. And as LibGDX is based on Java, we’ll keep using Java.

25 Feb

RenPy: Games, Rationale and The Power of a Well-Crafted Engine

More than 1 year after releasing NagiQ, I’d like to share a few bits of our experience with NagiQ, and of course, with the engine we used to build it: RenPy. First, we want to express our gratitude to RenPy’s author, PyTom, and to all of the RenPy contributors, including the kind people of the RenPy forums, who are always ready to provide useful feedback and answer our questions.

RenPy is a wonderful engine. What I like the most about RenPy is its robustness and cross-platform wonders. It feels like it might run anywhere and without ever crumbling. In fact, we released NagiQ for Windows, Mac and Linux, and deployment was incredibly easy: RenPy smoothly abstracted the differences among operating systems, and the game runs and behaves exactly the same way on all these 3 platforms. Besides, I think that several games built with RenPy have found their way into Steam and Google Play. I especially recommend RenPy for people creating its first game (NagiQ, developed in 2011, was our first title), because RenPy allows you to focus on what really matters for your game (gameplay, spaces, visuals and story.) As suggested, abstraction is a key concept for RenPy: abstraction is everywhere, it’s high-level and indeed powerful. For instance, right now I’m thinking of RenPy’s ATL, which is a very powerful way for showing displayables and applying several visual transformations (rotation, zoom, etc.) to them. And that’s sweet, really sweet. More so if you’re submitting games to publishers for the first time.

I don’t know of an engine better than RenPy for creating visual novels, and generally speaking, games in which pairs (image, text) are first-class citizens for scene design. NagiQ isn’t a visual novel, but a word game. As a word game, NagiQ is mostly based on simple images and strings. Handling images and strings with RenPy is a breeze. And not just strings. In RenPy you can show images and play music with a single line of code. A single line. You work fast, and you work safe. Further on, you might apply a tiny bit of ATL for achieving a richer presentation.

As RenPy is open-source, you can browse and study its code. You’ll end up learning quite a lot. Furthermore, knowing how things are done internally will allow you to find pathways for implementing any specific feature you might need. For example, several publishers require special handling of the quit message (Alt+F4 on Windows, CMD+Q on Mac OS X.) By knowing RenPy internals you’ll be able to handle this message as needed.

Overall, organizing your game’s scenes in RenPy is pretty straightforward. And you can even use neat transitions between scenes with a single line of code too. Normally, you’ll want a splash scene, a main menu scene, a level selection scene, and of course, the scenes implementing your main gameplay. Several of these scenes might require a lot of UI components whose creation is a piece of cake with RenPy. You’ll have the valuable help of Screens and Screen Language, which will simplify implementation and handling of your game’s screens.

RenPy is extensible. For the levels of NagiQ we required boards made of clickable cells. For doing that, we created a custom Board class in Python. And, again, RenPy proved golden: you can extend RenPy as you wish. We coded our Python class and integrated it to a RenPy scene pretty easily.

Summarizing, RenPy promotes creativity. It’s a very flexible and dynamic system, which doesn’t impose tight restrictions on the way you show and structure your content. The powerful ATL and the Screen Language are dreamy features, an outstanding support for unbounded creativity.

In retrospective, RenPy was indeed the best choice for building NagiQ: it was such a fun experience. And that’s what really matters. Long life to RenPy.

21 Feb

A few remarks on LWJGL for OS X and LibGDX

This little post is an answer to a comment in Hacking LWJGL, LibGDX and The Rainbow Machine. By the way, please note that the workarounds discussed on such post are not needed anymore, as LWJGL team has fixed the reported issues.

Warning

LWJGL for Mac OS X (using Java 7) is still labeled as experimental. It might contain bugs. As of this writing, there is no official release yet. Thereby, please be very careful if you’re planning to use this experimental branch, and test your game thoroughly. Read this thread to understand what’s the current status of this branch.

Where to get the experimental build for LWJGL?

In the above referred thread you can find the germane links.

Using this experimental branch with LibGDX

Download the experimental branch of LWJGL as explained above. Use the latest binaries, or build it from sources. Now there are several ways to update your LibGDX. What I recommend is to download the sources of the latest version of LibGDX and build it. Before building, update these files:

lwjgl.jar located in libgdx/backends/gdx-openal/libs/
gdx-backend-lwjgl-natives.jar located in libgdx/backends/gdx-backend-lwjgl/libs/

That should suffice. Remember, though, it’s an experimental branch. It might contain bugs.