Managing builds with premake

Premake

The branch I’m working with is available at https://bitbucket.org/Sairony/sairon-premake-stable/overview, not everything here applies to the official Premake version. My branch is very untested and have only been verified to work with our premake scripts for Visual Studio 2008, it probably works with 2005 as well although I haven’t tested.

Sorry about the formatting for the code, as the width of the blog makes it rather cramped I’d advice to mouse over the code and then pick ‘view source’ from the menu bar that appears to avoid having to scroll. First lets set up a solution which I’ll use as a template for my other 2 solutions:

solution "Common" -- Name for the solution, we'll only use this one as a template so won't be generated because there's no projects inside it
	configurations { "Debug", "Release", "ReleaseCandidate" } -- List of all our build types
	platforms { "Native", "PS3" } -- Platforms, Native is Win32 in our case

	configuration{ "*" } -- Selects all possible configurations
		defines { "BOOST_ALL_NO_LIB" }

	configuration { "Debug" } -- Will select Win32 | Debug & PS3 | Debug in our case
		defines { "_DEBUG" }
		flags { "Symbols" } -- Debug symbols

	configuration { "Release*" } -- Selects both Release & ReleaseCandidate
		defines { "NDEBUG", "_SECURE_SCL=0" }
		flags { "Optimize" } -- Optimization switches on

	configuration { "Native" } -- On Win32
		defines{ "WIN32" } -- Define WIN32

	configuration { "ReleaseCandidate" } -- All ReleaseCandidates needs a special define
		defines { "RELEASE_CANDIDATE" }

Most of this should be fairly straight forward. We create a solution, specify a couple of build types & platforms. The platform identifiers do have special meaning, so the allowed set needs to be extended unless yours are already supported. Inside Visual Studio there will be 6 different build configurations in the drop down list as the defined configurations{} & platforms{} are combined. Lets get on to actually defining a solution and a project, in this case Altus, our engine layer:

solution( "Dependencies", "Common" ) -- New solution, inherits values from "Common" defined above
	project "Altus" -- Creates & activates project Altus
		kind "StaticLib"
		language "C++"
		location "../src/Altus" -- Location of this projects file in relation to this script
		files { location() .. "**.h", location() .. "**.cpp", location() .. "**.hpp" } -- We need to append location(), as premake mostly uses the current script as base when specifying paths
		pchsource "PCH.cpp" -- Precompiled header implementation file
		pchheader "PCH.h" -- Precompiled header header
		forceincludes "PCH.h" -- Force includes the precompiled header across all source files
		-- Some file filters to tidy up the project in the generated project file. We actually need to nest 1 extra level ( the extra {} ) as premake doesn't support associative keys at top level.
		-- Otherwise the grouping is arbitrary. The order is important as filters be tested front to back removing files from the set as they're matched. Any file that doesn't pass a filter is placed
		-- at top level. By using / it's possible to create nested filters.
		filefilters{ {["Steam"] = "Steam/"},{["PS3"] = "PS3/"},{["Win32"] = "WIN32/"},{["Physics"] = "Physics", ["Physics"] = "Physx"}, { ["Serialization"] = "Serialization" }, {["Components"] = "Component\."} }
		defines { "AKSOUNDENGINE_STATIC", "BOOST_NO_ZLIB_AUTO" } -- Some defines & include directories for all configurations
		includedirs{ location() .. "/.", location() .. "/../boost", location() .. "/../loki/include" }

		configuration{ "not PS3" } -- For all builds that aren't PS3, exclude the PS3 sub folder from the build
			configexclude { "PS3/" }

		configuration{ "not Native" } -- For all Win32 builds exclude Steam & WIN32 folders
			configexclude { "WIN32/", "Steam/" }

		configuration{ "PS3" } -- PS3 includes
			includedirs{ ... }

		configuration{ "Native" } -- Win32 includes
			includedirs{ ... }

		configuration { "Release or ReleaseCandidate" }
			defines { "AK_OPTIMIZED" }

The Dependency solution actually has a ton of projects to it, some I can’t show and they all mostly use pretty much the same constructs so I’ve omitted them. I’ve also omitted a lot of the include directories as they aren’t really showing anything interesting. As the selection process for adding files uses wildcards to match any .h, .cpp or .hpp, even going into sub folders ( that’s the double * ), adding new files to the project inside Visual Studio will automatically include them in future generations of the the project. While it would seem solution{}, project{} & configuration{} introduces a scope that’s not actually true, tabs are only included for readability. Because of this it’s usually best to apply everything which applies to all configurations before activating custom configuration filters. Here follows another example, it’s a continuation of the Dependencies solution so it’s appended directly bellow the previous block. It shows the strength of the fact that premake actually uses a scripting language even if it mostly looks like data definitions:

	project "FreeType"
		kind "StaticLib"
		language "C++"
		location "../src/freetype"
		includedirs { location() .. "/include" }
		defines{ "FT2_BUILD_LIBRARY" }
		buildincludefilter "not PS3" -- Skip on PS3, only tools which are used on Win32 use FreeType
		local basedir = location() .. "/src/base/" -- For brevity
		-- Freetype ships a lot of files which shouldn't be built, as it turns out there's actually more files which shouldn't be built so it's easier to name them one by one instead of excluding unwanted
		files { basedir .. "ftbase.c", basedir .. "ftbbox.c",basedir .. "ftbase.c",  basedir .. "ftbitmap.c", basedir .. "ftdebug.c", basedir .. "ftfstype.c", basedir .. "ftgasp.c", basedir .. "ftglyph.c",
			basedir .. "ftinit.c", basedir .. "ftstroke.c", basedir .. "ftpfr.c", basedir .. "ftxf86.c", basedir .. "ftwinfnt.c", basedir .. "fttype1.c", basedir .. "ftsystem.c", basedir .. "ftsynth.c",
			basedir .. "ftpatent.c", basedir .. "ftotval.c", basedir .. "ftmm.c", basedir .. "ftlcdfil.c", basedir .. "ftgxval.c", location() .. "/src/winfonts/winfnt.c", location() .. "/src/cid/type1cid.c",
			location() .. "/include/freetype/config/*.h",  location() .. "/include/*.h" }
		-- FreeType uses a system where there's folders inside the src directory for most plugins ( there's exceptions, seen above ), we can iterate over these and add them programmatically
		local dirs = os.matchdirs( location() .. '/src/**' ) -- Get all sub folders
		for _, dir in pairs( dirs ) do
			local topFolder = dir:match(".*/(.*)") -- Get top level folders
			local filepath = dir .. '/' .. topFolder .. ".c" -- Lets see if there's a .c file with the same name as the folder
			if io.open( filepath ) then
				files{ filepath } -- Apparantly there was, so add it to the file set
			else
				filepath = dir .. '/ft'.. topFolder .. ".c" -- Some plugins have ft as prefix, so try with that instead
				if io.open( filepath ) then
					files { filepath }
				end
			end
		end
		excludes{ location() .. "/src/tools/**" } -- Tools are not to be built

		configuration { "Debug" }
			defines{ "FT_DEBUG_LEVEL_ERROR", "FT_DEBUG_LEVEL_TRACE" }

The neat part here is that we can include all plugins programmatically instead of having to list them manually. Some libraries depends on data in files which often might be needed as a part of the build process, this is rather easy in premake to just open the file up and retrieve it as a part of the generation process. Yet another example, this time showing custom build rules for specific files:

	project "PSSGCoreD3D"
		kind "StaticLib"
		language "C++"
		buildincludefilter "not PS3" -- No Directd3D on PS3
		location "..."
		includedirs { ... }
		files { location() .. "/*.cpp", location() .. "/*.h", location() .. "/*.vp", location() .. "/*.fp" } -- Vertex & fragment programs as part of the build

		configuration { "**.fp" } -- For all configurations let the following apply to files that ends in .fp
			buildcommands { '"CompileUtilityD3DFragmentShader.bat" $(InputName) $(IntDir)' }
			builddescription 'Compiling shader $(InputFileName) and creating object from HLSL file'
			buildoutputs { "$(IntDir)/$(InputName).fp.obj" }

		configuration{ "**.vp" } -- Different process for vertex programs
			buildcommands { '"CompileUtilityD3DVertexShader.bat" $(InputName) $(IntDir)' }
			builddescription 'Compiling shader $(InputFileName) and creating object from HLSL file'
			buildoutputs { "$(IntDir)/$(InputName).vp.obj" }

By utilizing the Visual Studio macros together with premake we can pass pretty much any data which could be meaningful to external tools for the build process this way. At last we run a fixup step where we apply rules to every solution:

print( "Fixup..." )
for sln in premake.solution.each() do -- Iterate over every solution
	solution( sln.name ) -- Activate it
	for _, prj in ipairs( sln.projects ) do -- Now iterate over every project within that solution
		if prj.solution == sln then -- Skip external projects ( it's possible to import the same project into multiple solutions with importproject() )
			if prj.name:find( "PSSG", 1, true ) == 1 then -- For a set of libraries we use which all have the prefix PSSG
				project( prj.name ) -- Activate that project

				configuration( "Debug" )
					targetname( prj.name .. "D" ) -- In debug builds it needs a capital d appended to the output name
			end
			project( prj.name )

			configuration{ "*App" }
				targetdir( "../bin/" .. prj.name ) -- All application variants should be output to the bin folder followed by the project name
		end
	end
	for _, cfg in ipairs(configurations()) do -- Now lets cover all configurations{}
		for _, plat in ipairs(platforms()) do -- Combined with platforms{}
			for _, prj in ipairs( sln.projects ) do -- We need to iterate at project level as well to activate the project
				if prj.solution == sln then -- Skip external projects
					local lib_path = path.join( "../lib" , cfg .. plat )
					project( prj.name ) -- Activate project so that location() gives us the correct info

					configuration { cfg, plat } -- Activate the config / platform combo
						libdirs { lib_path } -- And set the library search folder
						objdir( location() .. "/obj/" .. cfg .. plat ) -- Also create a folder inside that project for object files

					configuration { "*Lib", cfg, plat } -- All projects which output a library of some sort ( could be either dynamic or static )
						targetdir( lib_path ) -- Set the library output path
				end
			end
		end
	end
end
print( "Fixup done" )

This applies the directory scheme we use to make sure that everything points to correct versions of all the libraries. We have all of our solutions in the same script, but by using premakes variant of dofile() it’s easy to have a script per project which you can search for easily with the built in io functions.

The gains are numerous, first of all we can generate build files for everything which premake supports ( xcode, make, codeblocks, vs & codelite as of now it would seem ). As our branch is modified we’d need to do some fixing of the generators, but I’d say that’s a few hours top going by how little of the extensions we’ve added is dependent on the Visual Studio generator.

If we want to add lets say DLL builds I’d add 3 new items to configurations{} ( for Debug, Release & ReleaseCandidate ), I’d add a define for DLL export ( configuration{ “SharedLib” } to select only the libraries ) and a matching import for applications ( configuration{ “*App” } ). I’d then programatically at the end just set the kind to “SharedLib” for DLL configs for projects which already have “StaticLib” and fix the output & include paths, all in all a few lines. This process would’ve been tedious as hell if we’d have to do it inside Visual Studio for all of our projects.

One of the largest gains is that the code base is both fairly small and extensible, for example when I added the force include I opened a .vcproj with the parameter already set and noticed the output for it being ForcedIncludeFiles inside the compiler tool. I’d then open up premake/src/base/api.lua and add a new block for it in premake.fields with scope as config ( the pattern is easy to see ). Then I went over to the generator for the vcproj over at src/actions/vstudio/vs200x_vcproj.lua and went to the function vs200x_vcproj_VCCLCompilerTool and slammed in the following:

		if #cfg.forceincludes > 0 then
			_p(4,'ForcedIncludeFiles="%s"', premake.esc(path.translate(table.concat(cfg.forceincludes, ";"), '\\')))
		end

Basically, if there’s any items in the forceincludes member of the configuration, then escape and concatenate all those according to Visual Studios preferred format, and that’s it.

To generate your files is then just a matter of running premake on the command line: “pathtopremake –file=”rootscript.lua” vs2008″ will generate Visual Studio 2008 files. This counts as an action in premake, it’s possible to add your own, perhaps you want an action which builds all the generated solution files.

If it looks interesting you should check out their tutorial and reference, there’s also a forum.

Pages: 1 2
by / June 16, 2011 / Posted in: Development, Technical
Comments