In|Framez® Papers

dotXSI Programming – Shader Maker

By Wessam Bahnassi


This tutorial is divided into the following sections:


Welcome…

So you’re back for more dotXSI programming. In the previous tutorial, we saw together how we can read scene information from Softimage dotXSI files into our games. This time, we’ll extend our knowledge to discover more of the potentials hidden in this file format and its powerful API, the FTK.

Enter dotXSI... With Real-Time Shaders Support



Requirements




Overture…

Oh we have a lot of material to cover this time. And I do mean material! The final application will be able to open dotXSI scenes and read static shapes from it, along with their attached real-time shaders. This implies that we must have some flexible system that can handle the many types of shaders which we want to open. Of course, we’re not gonna handle all the real-time shader nodes that XSI provides. Instead, we’ll see how to open three of DSK|ShaderBass’s real-time nodes. These are DSK3DRenderer, DSK3DTextureStage and DSK3DParamTexture, allowing us to flexibly express conventional materials, plus some fancy blending and texturing modes. In addition, we’ll be covering some new nifty FTK tricks that will make our life much easier.
Good news is, this time we’ll be using the Semantic Layer to extract our scene data. In the previous tutorial, we learned how to use the low-level I/O Layer to read in the scene elements. Now, that tutorial, coupled with this one, should cover up the important aspects of using the two APIs, so you can choose the one that suits you better.

Ready for action?
Let’s begin…


Prelude to the FTK’s Semantic Layer

As of dotXSI 3.6, the FTK ships with a new API called the Semantic Layer. This layer is built on top of the core I/O Layer, to deliver high-level access to the elements expressed in the file.
The Semantic Layer is composed of many C++ classes that all derive from the base pure virtual CSLTemplate class. At the heart of CSLTemplate, lies a pointer to the corresponding CdotXSITemplate that this Semantic Layer class represents. This is good, because in the end, all what the Semantic Layer does, is just to wrap each template with an interface that allows us to gather the template’s info in a much more elegant way. This means that there is no data duplication happening inside, though some classes actually do contain more than just the CdotXSITemplate pointer, but this isn’t our concern. We know that the Semantic Layer (SL for short) is thin, and that it won’t make our code take twice the memory it should take.
Rarely, you’ll need to access a template using the I/O Layer within your SL code. In such cases, the CSLTemplate::Template() function will come to rescue. This function returns a pointer to the underlying CdotXSITemplate so you can get back to the I/O Layer at any time.

The sample application’s code now has been fully rewritten with the Semantic Layer, so you can open the I/O Layer version and the SL version for comparison.
Let’s see what happened to the project’s files.

Projects using SL differ a little from those that use the I/O Layer only. Properly including SL will also include the I/O Layer by the way.
First, there is this one and only header that we must include to gain access to the magic thing. Now, we do this in EnterDotXSI.h: 

#define __XSI_APPLICATION_
#define XSI_STATIC_LINK
#pragma warning(disable: 4100 4511 4505)
#include <SemanticLayer.h>
#pragma warning(default: 4100 4511)

Note how we #define XSI_STATIC_LINK prior to including SL’s header. Without this flag, you’ll get a helluva compiler errors, so this one is necessary now.
Well, that’s all about it. We’re now capable of reading dotXSI files using SL!
Just one more thing, we’ll #include <list> so we have access to STL’s list and string classes. We’ll use this to overcome the limitation of the static number of entities that we can load. It’s just a couple of lines in the header: 

#include <list>
#include <string>
using namespace std;

That’s it…


Silk and Steel… Converting EnterDotXSI from the I/O Layer to Semantic Layer

In order to save time and space, I won’t go listing each and every difference between I/O and SL projects. However, the function CMyD3DApplication::OpenDotXSIFile() requires some explanation.
Here it is in its new suit: 

BOOL CMyD3DApplication::OpenDotXSIFile(LPSTR szFileName,LPSTR szFileTitle)
{
	eSI_Error siError=SI_SUCCESS;	// To hold error codes

	CSLScene xsiScene;

	// Open the xsi file
	siError = (eSI_Error)xsiScene.Open(szFileName);
	if (siError != SI_SUCCESS)
	{
		DXTRACE("ERROR: Failed to open file %s...\n",szFileTitle);
		return FALSE;
	}

	// Attempt to read the file... Save the result in our error code variable...
	siError = (eSI_Error)xsiScene.Read();
	if (siError != SI_SUCCESS)
	{
		DXTRACE("ERROR: Failed to read dotXSI file: %s\n",szFileTitle);
		return FALSE;
	}

	// Dump file info
	DXTRACE("INFO: %s header information:\n",szFileTitle);
	DXTRACE("      File version : %d.%d\n",xsiScene.Parser()->GetdotXSIFileVersionMajor(),
					xsiScene.Parser()->GetdotXSIFileVersionMinor());

	CSLFileInfo *pFileInfo = xsiScene.FileInfo();
	if (pFileInfo)
	{
		DXTRACE("      Project Name: %s\n",pFileInfo->GetProjectName());
		DXTRACE("      User Name: %s\n",pFileInfo->GetUsername());
		DXTRACE("      Last Time Modified: %s\n",pFileInfo->GetSaveDateTime());
		DXTRACE("      Built in: %s\n",pFileInfo->GetOriginator());
	}

	dotXSILoadMeshes(xsiScene.Root());

	DWORD dwMeshesCount = 0;
	XSIMESHESLIST::iterator itMeshes = m_XSIMeshes.begin();
	while (itMeshes != m_XSIMeshes.end())
	{
		XSIMESH *pMesh = *(itMeshes++);
		if (pMesh->pMesh)
			dwMeshesCount++;
	}
	DXTRACE("Successfully loaded %u models with %i meshes\n",m_XSIMeshes.size(),dwMeshesCount);

	return TRUE;
}

Huh! Where did the whole CXSIParser object go?!
It’s now substituted by the single CSLScene instance. The CSLScene object holds information about the whole dotXSI scene, and it’s the object from which we’ll access all scene elements.
One of the first elements that we can directly access from CSLScene is CSLFileInfo. Clearly, CSLFileInfo represents the SI_FileInfo template. So, we can use this interface to access that template’s parameters. Note how we retrieve the object from CSLScene and check for its validity. This is important because there’s a chance that this dotXSI file doesn’t contain an instance of SI_FileInfo, thus you can’t access it. This behavior is common to all other SL objects that return references to (yet) other SL objects.
One line that demands explanation too is the File Version trace line: 

DXTRACE("      File version : %d.%d\n",xsiScene.Parser()->GetdotXSIFileVersionMajor(),
				xsiScene.Parser()->GetdotXSIFileVersionMinor());

Needless to say, DXTRACE works much like printf() but logs to the debug output.
What I wanted to show is that the CSLScene object doesn’t directly expose the file’s version information, though we used to get it from CXSIParser. So, we can directly go to the I/O Layer by asking CSLScene to return the internal CXSIParser object. Once that’s done, we can get the file version in the same old way. Cool isn’t it?
Next, comes a single call to a new function called dotXSILoadMeshes(). This function barely replaces our ParseTemplates() function in the I/O Layer version. It’s a recursive, and starts its recursion from the scene’s root model, going down through all children models that contain meshes. We’ll see that shortly.
Note that in this version, I used STL’s list template class to hold a list of the meshes we load, instead of the old version’s static-size array. The meshes list is declared in CMyD3DApplication. In the code above, we iterate this list to count the number of valid meshes that we loaded, so we can display this information in the debug output.

Still easy.

Now let’s get a quick skim over dotXSILoadMeshes(): 

eSI_Error CMyD3DApplication::dotXSILoadMeshes(CSLModel *pRootModel)
{
	// Load the meshes
	if (!pRootModel)
		return SI_ERR_INVALID_SCENE;

	// Get the children
	CSLModel **ppModels = pRootModel->GetChildrenList();
	int iChildrenCount = pRootModel->GetChildrenCount();
	for (int i=0;i<iChildrenCount;i++)
	{
		CSLModel *pThisModel = ppModels[i];

		// Only interested in meshes
		if (pThisModel->GetPrimitiveType() == CSLModel::SI_MESH)
		{
			// Load the model
			if (dotXSILoadModel(pThisModel) == SI_SUCCESS)
				dotXSILoadMesh((CSLMesh*)pThisModel->Primitive());	// Load the mesh
		}

		// Recurse for children
		dotXSILoadMeshes(pThisModel);
	}

	return SI_SUCCESS;
}
The function is a recursive that takes a single CSLModel pointer as a parameter. CSLModel mostly describes SI_Model templates, but that’s not always the case. Remember that we’re talking semantically. A CSLModel can only have children CSLModels. So, what about other entities? Where did meshes, lights, cameras …etc go? How can we access meshes that are children of an SI_Model?
The answer lies in the CSLModels::Primitive() function. A CSLModel is always bound to one primitive a time. A primitive can be any entity placed under SI_Model (e.g. SI_Mesh, SI_Light).

Ok, this is great, now how can I know what type of primitive this CSLModel expresses?
Yep, it’s CSLModel::GetPrimitiveType(). This function returns a value describing what primitive lies under. Right now, we’re interested in CSLModel::SI_MESH primitive types. Once we know that we’re dealing with a CSLModel::SI_MESH primitive, we can happily typecast the CSLPrimitive pointer that’s returned by CSLModels::Primitive() to a pointer of type (CSLMesh*). We do this when calling dotXSILoadMesh(), which is the function that actually reads the mesh’s vertices and builds the internal D3D object for rendering.
I won’t list dotXSILoadMesh() here because its too lengthy, so you can go ahead and read it from the project’s source code. It’s clear and well commented.


There in The Silence… User Data

What user data? I’ll be introducing a nifty technique that will save us a lot of hassle when binding entities together. User Data is a list of small fields found on all CSIBCNode descendants (and all SL classes are inherited from CSIBCNode). These fields don’t contain anything meaningful to the FTK. They’re just for you! Fill’em, zero’em, leave’em… It just doesn’t matter. Since these fields are present with each unique CSIBCNode instance, then we can use them to attach custom data to any FTK object (including many I/O Layer objects).

Before going on, allow me to list the APIs needed to deal with custom data:

Alright, now let’s see how the old EnterDotXSI used to bind a mesh to its parent model:

Now, I’ll list the corresponding method of access after adopting User Data and SL:

Notice how we avoided many clunky search operations. This leads to much faster and reliable code. For the matter, I added a new function that safely returns user data. Let’s look at it: 

void* CMyD3DApplication::GetTemplateUserData(CSLTemplate *pTemplate)
{
	if (!pTemplate)
		return NULL;

	CSIBCArray<CSIBCUserData*>& udList = pTemplate->UserDataList();
	if (udList.GetSize() == 0)
		return NULL;

	return udList[0]->GetData();
}

Without this, you can easily be asking for a non-existing User Data field, and thus, asking for a potential crash.
Haven’t you noticed? We’re only dealing with the first element of the User Data list. This is specific to our application’s case, as it doesn’t make use of more that one field per instance.
By the way, this technique can be successfully applied with the I/O Layer too.

That’s one more thing off. Through the rest of this tutorial, we’ll make good use of User Data in much the same way I described above to bind different entities together.
Take your time to read the SL version of EnterDotXSI and understand how things changed. The mesh loading code has changed a little bit, but it’s very clear.


Come As You Are… Mimicking XSI Materials

Ah, finally. We arrived to the main subject of this tutorial: reading XSI materials. This is an easy task, provided that you have an extensible system that can handle things out. Unfortunately, we don’t have it! So, we’re gonna make it.

Our internal representation of an XSI material is very simple: a string that identifies the material (its name), and a shaders tree.
Don’t panic! In XSI, Real-time shader nodes can only have one input port, thus, a Real-time shader tree will always be a list of nodes connected serially to each other. Because of this design decision, we can express this tree with a simple STL list container.

So, our material class is now declared as follows: 

class CXSIMaterial
{
public:
	~CXSIMaterial();

	DWORD GetNumPasses(void);
	void Pass(DWORD dwPass);
	void PassDefaults(void);
	XSISHADERSLIST* GetShaders(void) { return &m_lstShaders; };
	string& GetName(void) { return m_strName; };

protected:
	string m_strName;
	XSISHADERSLIST m_lstShaders;
};

Simple enough, eh? Beside the mutators, there are some other important public methods here. Those deserve some explanation.
Some materials require that the object gets rendered on more than one render pass. Our material class is capable of this. To get the number of passes this material requires, just call CXSIMaterial::GetNumPasses(). This runs a query on the underlying shaders list to see the maximum passes they require.
The other two functions are CXSIMaterial::Pass() and CXSIMaterial::PassDefaults(). The former applies the shaders’ states to the rendering device, so the next rendering call will be affected by those states.
What CXSIMaterial::PassDefaults() does is return the rendering device to its original state after it has altered it with the call to Pass(). This is important if we want to avoid unexpected states left by previous materials.
Finally, there’s this m_lstShaders member variable. This is an ordered list of connected shaders. The order is the same as the one in XSI, first node is that which directly connects to the material’s ‘Realtime’ port.
But, what objects are contained in this list? We’ll see that in the next section.


Infinite Dreams… Coding an extensible Real-time Shader System

What should we expect from our shader system?
Some easy unified interface that allows us to correctly apply shaders. It also has to be extensible. Great, sounds like a big game of polymorphism here.
We’ll express each shader node with a single object instance. The behavior of the node will be decided by deriving from a base class that provides the required base services.
This base class is CXSIShaderNode, and we’ll declare it as follows: 

class CXSIShaderNode
{
public:
	CXSIShaderNode(LPDIRECT3DDEVICE9 pDevice);
	virtual ~CXSIShaderNode() {};

	virtual DWORD GetNumPasses(void) { return 1; };
	virtual void Pass(DWORD dwPass) = 0;
	virtual void PassDefaults(void) {};

	void SetPreviousNode(CXSIShaderNode *pShader) { m_pPreviousNode = pShader; };
	void SetNextNode(CXSIShaderNode *pShader) { m_pNextNode = pShader; };

	CXSIShaderNode* GetPreviousNode(void) const { return m_pPreviousNode; };
	CXSIShaderNode* GetNextNode(void) const { return m_pNextNode; };
	string& GetName(void) { return m_strName; };

protected:
	string m_strName;
	CXSIShaderNode *m_pPreviousNode;
	CXSIShaderNode *m_pNextNode;
	LPDIRECT3DDEVICE9 m_pD3DDevice;
};

CXSIShaderNode’s constructor takes a Direct3D9 device object as a parameter. This is important because a shader needs the device on which it can apply its settings on. So this is shared in all of our shaders.
A CXSIShaderNode also has a name, and two pointers to other CXSIShaderNode objects. These are the ‘previous’ and ‘next’ nodes connected to this one. We’ll use this info when navigating the render tree in CXSIMaterial.
There’s a bunch of virtual functions that can be overridden to reflect specific shader behavior. Note that CXSIShaderNode::Pass() is a pure-virtual function, which means that it must be overridden.
This function takes one parameter, which is the number of the pass we’re currently rendering. If the shader node doesn’t provide such a number of passes, it should return silently.

Both CXSIMaterial and CXSIShaderNode are declared in EnterDotXSI.h, and they’re implemented in EnterDotXSI.cpp. I won’t list the implementation because it’s very easy and clear. I just want to note that CXSIMaterial::Pass() iterates the shaders list from the end to the start. This is close to what XSI does when evaluating real-time shader nodes. The final node that gets evaluated is the one directly connected to the material’s ‘Realtime’ port.


Can I Play With Madness… Implementing DSK3DRenderer

We’ll instantly put our shader class into business by implementing the DSK3DRenderer shader node.
As I said, DSK3DRenderer is a node from DSK|ShaderBass®. You gotta take a live look at it before you can implement it. In the attached project’s directory, you’ll find a couple of files that contain DSK3DRenderer in their materials. Take ‘statue.xsi’ for example, and open it with XSIDump. We have to use XSIDump because it provides type information, which is very important to us here, especially if we don’t want to do SPDL hacky wackies.
Note what information does a DSK3DRenderer provide. We won’t read all of it, simply because some of the fields require specific engine support. We’ll be reading most of DSK3DRenderer’s Conventional Properties, but we won’t touch the Effect Properties.
Ok, the fields we’re gonna tackle express simple states that can be directly set on the Direct3D device. So, we’ll create a new header ‘DSK3DRenderer_Shader.h’ which will contain the declaration of our DSK3DRenderer class. This is what the final class looks like: 

class DSK3DRenderer : public CXSIShaderNode
{
public:
	DSK3DRenderer(LPDIRECT3DDEVICE9);

	DWORD GetNumPasses(void) { return 1; };
	void Pass(DWORD dwPass);
	void PassDefaults(void);

	D3DMATERIAL9 m_Material;
	DWORD m_ShadeMode;
	DWORD m_FillMode;
	BOOL m_Lit;
	BOOL m_AlphaBlended;
	DWORD m_SrcBlend;
	DWORD m_DestBlend;
	DWORD m_BlendOp;
	DWORD m_CullMode;
};

The fields are very D3D-specific, so I won’t go into their details, but they’re very self explanatory. The ‘m_Material’ member variable is of type D3DMATERIAL9, which is a structure that contains Diffuse, Specular, Emissive, and Ambient color values for the material. If you’re interested in the internal implementation of this shader, open ‘DSK3DRenderer_Shader.cpp’ and check the Pass() and PassDefault() functions.


Putting all eggs in one basket… Loading the material library

Ok, now that we can read some type of shaders, let’s go head and write the corresponding FTK code.
In dotXSI, all materials in the scene are grouped under one SI_MaterialLibrary template instance, which contains one or more XSI_Material instances (it can contain SI_Material instances, but we’re skipping those since they can’t handle shaders). Each XSI_Material in turn contains one or more XSI_Shader instance. You can observe this clearly in ‘statue.xsi’. So, all what we have to do is just load in this material library, then have our meshes select the material that suits them from the library.

And so we write CMyD3DApplication::dotXSILoadMaterialLibrary().
Following is the function’s implementation: 

eSI_Error CMyD3DApplication::dotXSILoadMaterialLibrary(CSLScene *pScene)
{
	CSLMaterialLibrary *pMatLib = pScene->GetMaterialLibrary();
	if (!pMatLib)
		return SI_ERR_NO_MATERIAL;

	DXTRACE("XSI Material Library contains %i materials\n",pMatLib->GetMaterialCount());

	CSLBaseMaterial **ppMaterials = pMatLib->GetMaterialList();
	CSLXSIMaterial *pXSIMaterial;
	for (SI_Int i=0;i<pMatLib->GetMaterialCount();i++)
	{
		if (ppMaterials[i]->Type() != CSLTemplate::XSI_MATERIAL)
			continue;	// We only handle XSI_Material templates
		pXSIMaterial = (CSLXSIMaterial*)ppMaterials[i];

		// Create the material object
		CXSIMaterial *pMat = new CXSIMaterial;
		if (!pMat)
			return SI_ERR_ALLOC_PROBLEM;

		// Add this object to the material template, so we can access it quickly
		pMat->GetName() = pXSIMaterial->GetName();
		pXSIMaterial->AttachUserData("MaterialObject",pMat);

		// Add the material to our library
		m_XSIMaterialLib.push_back(pMat);

		// Load the material's shaders
		CSLXSIShader **ppShaders = pXSIMaterial->GetShaderList();
		CSLXSIShader *pPrevShader;
		CXSIShaderNode *pNode,*pPrevNode;
		for (SI_Int j=pXSIMaterial->GetShaderCount()-1;j>=0;j--)
		{
			if (strstr(ppShaders[j]->GetProgID(),"DSK3DRenderer"))
				pNode = dotXSILoadDSK3DRenderer(ppShaders[j]);
			//else if (strstr(ppShaders[j]->GetProgID(),"DSK3DParamTexture"))
				//pNode = dotXSILoadDSK3DParamTexture(ppShaders[j]);
			//else if (strstr(ppShaders[j]->GetProgID(),"DSK3DTextureStage"))
				//pNode = dotXSILoadDSK3DTextureStage(ppShaders[j]);
			else continue;

			if (pNode == NULL)
			{
				DXTRACE("Failed to load shader %s\n",ppShaders[j]->Name().GetText());
				continue;
			}

			pNode->GetName() = ppShaders[j]->GetName();

			// Attach node to template
			ppShaders[j]->AttachUserData("ShaderObject",pNode);

			// Link it to its previous shader, if it exists
			pPrevNode = NULL;
			pPrevShader = ppShaders[j]->GetConnectionPointList()[0]->GetShader();
			if (pPrevShader)
				pPrevNode = (CXSIShaderNode*)GetTemplateUserData(pPrevShader);
			pNode->SetPreviousNode(pPrevNode);
			if (pPrevNode)
				pPrevNode->SetNextNode(pNode);

			// Add the shader to the material
			pMat->GetShaders()->push_front(pNode);
		}	// For each shader in the material
	}	// For each material in the library

	return SI_SUCCESS;
}

The code starts by retrieving the CSLMaterialLibrary object from CSLScene. If this pointer is ‘NULL’, then this scene doesn’t have a material library in it, and we just return an error describing the dangerous situation we're in.
We retrieve a list of the materials in the library by calling CSLMaterialLibrary::GetMaterialList(), which will return an array of CSLBaseMaterial pointers.

Why CSLBaseMaterial?
That’s because dotXSI 3.0 and lower versions use the older SI_Material template instead of the newer XSI_Material. We’re not interested in the older SI_Material here, because (as I said earlier) they cannot express the material’s render tree, which makes most of our tutorial useless!

And so, we loop on each material in the library, asking it if it was of the type CSLTemplate::XSI_MATERIAL, and skipping it if it’s not.
Next, we create our internal CXSIMaterial object, and save its pointer as a User Data field in the CSLXSIMaterial object. This is done so that meshes can directly connect to the correct material. Plus, we add the material to a list declared in our application. This list is our internal representation of the material library, and it’s very important for clean-up purposes.
Next, we get the list of shaders associated with this material. Shaders are identified by their ProgID, which follows the form ‘Softimage.ShaderName.1’. In the code above, we skip those nodes that are not DSK3DRenderers. Clearly, the function dotXSILoadDSK3DRenderer() takes a pointer to the corresponding CSLXSIShader, and returns a pointer to a CXSIShaderNode object. We’ll get into this function later.
The rest of the steps are easy. Add the object into CSLXSIShader’s User Data, and setup the connection between this node and its previous one. Because of this behavior, we iterate the shaders list from end to start. This ensures that newer nodes can directly connect to previous ones without an additional connection pass.
Now, let’s take a look at the implementation of CMyD3DApplication::dotXSILoadDSK3DRenderer(): 

CXSIShaderNode* CMyD3DApplication::dotXSILoadDSK3DRenderer(CSLXSIShader* pXSIShader)
{
	DSK3DRenderer *pRenderer = new DSK3DRenderer(m_pd3dDevice);
	if (!pRenderer)
		return NULL;

	pRenderer->m_Material.Diffuse.r = GET_XSIPARAM_FLOATVAL(pXSIShader,"Diffuse.red");
	pRenderer->m_Material.Diffuse.g = GET_XSIPARAM_FLOATVAL(pXSIShader,"Diffuse.green");
	pRenderer->m_Material.Diffuse.b = GET_XSIPARAM_FLOATVAL(pXSIShader,"Diffuse.blue");
	pRenderer->m_Material.Diffuse.a = GET_XSIPARAM_FLOATVAL(pXSIShader,"Diffuse.alpha");

	pRenderer->m_Material.Ambient.r = GET_XSIPARAM_FLOATVAL(pXSIShader,"Ambient.red");
	pRenderer->m_Material.Ambient.g = GET_XSIPARAM_FLOATVAL(pXSIShader,"Ambient.green");
	pRenderer->m_Material.Ambient.b = GET_XSIPARAM_FLOATVAL(pXSIShader,"Ambient.blue");
	pRenderer->m_Material.Ambient.a = GET_XSIPARAM_FLOATVAL(pXSIShader,"Ambient.alpha");

	pRenderer->m_Material.Emissive.r = GET_XSIPARAM_FLOATVAL(pXSIShader,"Emissive.red");
	pRenderer->m_Material.Emissive.g = GET_XSIPARAM_FLOATVAL(pXSIShader,"Emissive.green");
	pRenderer->m_Material.Emissive.b = GET_XSIPARAM_FLOATVAL(pXSIShader,"Emissive.blue");
	pRenderer->m_Material.Emissive.a = GET_XSIPARAM_FLOATVAL(pXSIShader,"Emissive.alpha");

	pRenderer->m_Material.Specular.r = GET_XSIPARAM_FLOATVAL(pXSIShader,"Specular.red");
	pRenderer->m_Material.Specular.g = GET_XSIPARAM_FLOATVAL(pXSIShader,"Specular.green");
	pRenderer->m_Material.Specular.b = GET_XSIPARAM_FLOATVAL(pXSIShader,"Specular.blue");
	pRenderer->m_Material.Specular.a = GET_XSIPARAM_FLOATVAL(pXSIShader,"Specular.alpha");

	pRenderer->m_Material.Power = GET_XSIPARAM_FLOATVAL(pXSIShader,"Power");

	pRenderer->m_ShadeMode = GET_XSIPARAM_INTVAL(pXSIShader,"Shading");

	pRenderer->m_FillMode = GET_XSIPARAM_INTVAL(pXSIShader,"Fill");

	pRenderer->m_Lit = GET_XSIPARAM_BOOLVAL(pXSIShader,"Lighting");

	pRenderer->m_AlphaBlended = GET_XSIPARAM_BOOLVAL(pXSIShader,"AlphaBlendEnable");

	pRenderer->m_SrcBlend = GET_XSIPARAM_INTVAL(pXSIShader,"SrcBlend");
	pRenderer->m_DestBlend = GET_XSIPARAM_INTVAL(pXSIShader,"DestBlend");
	pRenderer->m_BlendOp = GET_XSIPARAM_INTVAL(pXSIShader,"BlendOp");

	pRenderer->m_CullMode = GET_XSIPARAM_INTVAL(pXSIShader,"CullMode");

	return pRenderer;
}

Very straight forward! At the beginning we allocate a new DSK3DRenderer shader node, then we set its properties that are read from the dotXSI file. Because of the nature of the XSI_Shader template, the parameters are not strictly specified. Each shader may have its own number of parameters, thus there’s no way for CSLXSIShader to provide direct accessor functions to those dynamic parameters. They can change from shader to shader. So, there comes the generic CSLXSIShader::ParameterFromName() call. This function returns the parameter that matches the name specified. It returns a pointer of type CSLAnimatableType. This class provides the function CSLAnimatableType::GetFloatValue() which serves us good for floating-point parameters. But what about integer and boolean values?
If we cast this pointer to the more generic CSLVariantParameter type, we can access the parameter’s internal SI_TinyVariant value, from which we can access the type we need.

I’ve written some macros that will make this operation much cleaner. These are:
GET_XSIPARAM_FLOATVAL, GET_XSIPARAM_BOOLVAL, GET_XSIPARAM_INTVAL and GET_XSIPARAM_STRINGVAL.
All take the parameter’s owner object (the CSLXSIShader object) and a string containing the parameter’s name. Parameter names and types can be easily brought by XSIDump.

I have added a new member to the XSIMESH structure so now it can hold a reference to its material.
It now looks like: 

struct XSIMESH
{
	string strModelName;
	D3DXVECTOR3 vTranslation;
	D3DXVECTOR3 vRotation;
	D3DXVECTOR3 vScaling;
	D3DXMATRIX matWorld;
	LPD3DXMESH pMesh;
	CXSIMaterial* pMaterial;	// This mesh’s material
};

With the material library loaded, now we can safely add this line to the end of our mesh loading function CMyD3DApplication::dotXSILoadMesh(): 

// Attach the material to the mesh
pXSIMesh->pMaterial = (CXSIMaterial*)GetTemplateUserData(pTriList->GetMaterial());

Easy enough, eh? Since we store a pointer to our CXSIMaterial in CSLMaterial’s User Data, then it’s very straightforward to bind the mesh with CXSIMaterial. Just call CSLTriangleList::GetMaterial(), which will return a pointer to CSLMaterial that this tri-list is using. We pass this pointer to GetTemplateUserData() and cast the return value to (CXSIMaterial*), and bingo! We have our material set on the corresponding mesh!


This is The Time… Applying our shaders in rendering

We now have our material library loaded, and our meshes initialized. All what is left is to setup CMyD3DApplication::Render() to take advantage of the new material attached with each mesh. With our newly built material system, this process becomes very easy. This is the new rendering loop in CMyD3DApplication::Render(): 

// Traverse our meshes list, drawing each one
XSIMESHESLIST::iterator itMeshes = m_XSIMeshes.begin();
while (itMeshes != m_XSIMeshes.end())
{
	XSIMESH *pXSIMesh = *(itMeshes++);
	if (pXSIMesh->pMesh == NULL)
		continue;	// An empty model, skip it

	m_pd3dDevice->SetTransform(D3DTS_WORLD,&pXSIMesh->matWorld);
	if (!pXSIMesh->pMaterial)
	{
		pXSIMesh->pMesh->DrawSubset(0);
		continue;
	}

	DWORD dwPasses = pXSIMesh->pMaterial->GetNumPasses();
	for (DWORD i=0;i<dwPasses;i++)
	{
		pXSIMesh->pMaterial->Pass(i);
		pXSIMesh->pMesh->DrawSubset(0);
	}
	pXSIMesh->pMaterial->PassDefaults();
}

We added a check line that handles meshes without a material just for sanity. The important code is that which starts by taking the material’s number of passes, then loops on each pass. Inside the loop, we call CXSIMaterial::Pass() to apply the shaders for that pass, then we draw the mesh as usual. When we’re done with our passes, we call the material’s PassDefault() to return device states to their original settings.

Yep! Just that! Now we can open any mesh with a DSK3DRenderer shader attached to it!


Another Brick in The Wall… Adding DSK3DTextureStage & DSK3DParamTexture

Our lone implementation of the DSK3DRenderer shader doesn’t provide that much of options; just basic material and alpha blending options. We need something more visually appealing.
Perhaps you’ve already noticed that CMyD3DApplication::dotXSILoadMesh() is now aware of texture coordinate information. If it finds one, it will include it into our mesh’s buffer, but until now, we’re not using this information.
We’ll add texturing support by implementing two more shaders from DSK|ShaderBass®. These are DSK3DTextureStage and DSK3DParamTexture. You can read DSK|ShaderBass’s documentation about what options these two shader nodes provide.
In short, DSK3DParamTexture is responsible for specifying a texture file name to be loaded and used by the material. DSK3DTextureStage basically specifies what color operations to perform between this texture and other color inputs, as well as some other blending and UV mapping aspects.

The method we’ll use to open the DSK3DTextureStage shader node is very similar of that of DSK3DRenderer, just with different parameter names.
We add a loading function in our framework that handles those shaders creation. It’s CMyD3DApplication::dotXSILoadDSK3DTextureStage(). This function is called by CMyD3DApplication::dotXSILoadMaterialLibrary() when it faces a material that contains a DSK3DTextureStage shader node. You can take a look at our implementation of DSK3DTextureStage in the files ‘DSK3DTextureStage_Shader.cpp’ and ‘DSK3DTextureStage_Shader.h’.

Unlike DSK3DTextureStage, the DSK3DParamTexture shader node requires more explanation.
Here, I’m listing this shader’s parameters as displayed by XSIDump: 

(PCHAR) ParamName:
(PCHAR) FileName: skin.bmp
(INT) Stage: 0
(INT) Format: 0
(INT) MipMapsCount: 0
(BOOL) RenderTarget: 0

The first parameter ‘ParamName’ is of special use to DSK|ShaderBass’s Effect Renderer, so we won’t care about it here. Next, comes the ‘FileName’ parameter, which is very important to us here. The ‘Stage’ parameter allows us to setup a multi-texture cascade so we can achieve special blending effects. DSK3DParamTexture allows for 8 textures to be set at once on the rendering device, provided that the 3D adapter can handle such a capability.
Then we have the ‘Format’ parameter. This is an integer ID that tells us what format this texture should be loaded with. This number directly maps to the D3DFORMAT enumerated type. So, a format of 21 is D3DFMT_A8R8G8B8, while 22 is D3DFMT_X8R8G8B8 (no alpha). If the format is 0, then it’s up to us to decide what format is suitable for this texture. Don’t worry about this though.
The ‘MipMapsCount’ tells us how many Mipmap levels should we create from our texture. Again, a value of 0 is a special case in which we generate the whole Mipmap chain from the largest dimension down to 1x1.
The last parameter ‘RenderTarget’ informs us whether this texture should be used as a render target texture or not. We don’t make use of this feature here, so we’ll skip on it.

Let’s take a look at the class used to represent this node: 

class DSK3DParamTexture : public CXSIShaderNode
{
public:
	DSK3DParamTexture(LPDIRECT3DDEVICE9);
	~DSK3DParamTexture();

	DWORD GetNumPasses(void) { return 1; };
	void Pass(DWORD dwPass);
	void PassDefaults(void);

	BOOL CreateTexture(void);

string m_strFileName;
	DWORD m_Stage;
	DWORD m_Format;
	DWORD m_MipMapsCount;

protected:
	LPDIRECT3DTEXTURE9 m_pTexture;
};

The usual matching data fields are there as member variables. One member that seems interesting is ‘m_pTexture’. This is the actual D3D texture object that we’ll be creating from the file.
Also, there’s a new function amongst the overrides. DSK3DParamTexture::CreateTexture() will create a valid D3D texture out of the input parameters, and save its result in ‘m_pTexture’.

Let’s check out the implementation of this function, it contains something interesting other than pure D3D code: 

BOOL DSK3DParamTexture::CreateTexture(void)
{
	// Try loading with D3DX
	HRESULT hRetval = D3DXCreateTextureFromFileEx(
		m_pD3DDevice,	// Direct3D Device
		m_strFileName.data(),	// File Name
		D3DX_DEFAULT,	// Width
		D3DX_DEFAULT,	// Height
		m_MipMapsCount,	// MipLevels
		0,		// Usage
		(D3DFORMAT)m_Format,	// Format
		D3DPOOL_MANAGED,// Pool
		D3DX_DEFAULT,	// Filter
		D3DX_DEFAULT,	// MipFilter
		0,		// ColorKey
		NULL,		// Image Info
		NULL,		// Palette
		&m_pTexture);	// Output Interface

	if (SUCCEEDED(hRetval))
		return TRUE;

	// Is it a .pic file?
	CSIBCPixMap pixMap;
	CSIILPICFileDriver picDriver;
	pixMap.AddDriver(&picDriver);

	eSI_Error eRetval = (eSI_Error)CSIBCPixMap::Load(m_strFileName.data(),pixMap);
	if (eRetval != SI_SUCCESS)
	{
		DXTRACE("Can't load file %s\n",m_strFileName.data());
		return FALSE;
	}

	// Create the texture interface
	D3DXCreateTexture(
		m_pD3DDevice,	// Direct3D Device
		D3DX_DEFAULT,	// Width
		D3DX_DEFAULT,	// Height
		m_MipMapsCount,	// MipLevels
		0,		// Usage
		(D3DFORMAT)m_Format,// Format
		D3DPOOL_MANAGED,	// Pool
		&m_pTexture);	// Output Interface

	if (!m_pTexture)
	{
		DXTRACE("Can't create texture for file %s\n",m_strFileName.data());
		return FALSE;
	}

	// Load the image on the texture
	D3DXFillTexture(m_pTexture,FillPICTexture,(void*)&pixMap);

	return TRUE;
}
The first lines might look cryptic for you non-Direct3D programmers, but hey. The function D3DXCreateTextureFromFileEx() is a utility function that comes with the Direct3D Extensions library. It takes all the parameters we loaded from the dotXSI file, and magically returns an active D3D texture object.

Well?

One fact about D3DXCreateTextureFromFileEx() is that it just doesn’t support loading Softimage .pic files, which is very vital for us XSI users. So, we check if the D3DX function failed, then we roll our own .pic texture handler.
Luckily for us, the FTK provides a bunch of helpers that make loading .pic textures a very easy operation. Textures in the FTK (called pixel maps) are represented by the CSIBCPixMap class. The class on its own doesn’t do much. It actually can’t read any format, until you add a format ‘driver’ to it.
A format driver is a class inherited from CSIBCPixMapDriver, which provides specialized code to handle a specific file format. Amongst the formats available by the FTK, we’re interested mostly in the CSIILPICFileDriver, which allows CSIBCPixMap to load .pic files. So, we construct a CSIBCPixMap object, and register the CSIILPICFileDriver with it. You can register other drivers if you want the pixmap to handle mode formats.
Next, we call CSIBCPixMap::Load() to load the .pic file. This function is a static function in the CSIBCPixMap class, so it takes the texture file name plus a reference to an existing CSIBCPixMap object that will be filled with the texture data. Once that’s done successfully, we create an empty D3D texture object using D3DXCreateTexture().
This (again) takes the format and mipmap parameters into consideration. To fill the D3D texture with data from the pixmap, we use another helper function called D3DXFillTexture(). This function takes the address of a user-defined fill function (that we’ll see shortly) to fill the D3D texture object.
What’s cool about this function is that it automatically handles all the gory details of format conversion and mipmap regeneration. You just give it pixels in full RGBA color, and it saves it in the suitable format on the texture.

The user-defined callback function follows: 

void WINAPI FillPICTexture(D3DXVECTOR4* pOut,CONST D3DXVECTOR2* pTexCoord,CONST D3DXVECTOR2*,LPVOID pData)
{
	CSIBCPixMap *pPicPixels = (CSIBCPixMap*)pData;

	int iX,iY,iR,iG,iB,iA;
	iX = (int)((float)pPicPixels->GetWidth() * pTexCoord->x);
	iY = (int)((float)pPicPixels->GetHeight() * pTexCoord->y);

	pPicPixels->GetPixel(iX,iY,iR,iG,iB,iA);
	pOut->x = (float)iR/255.0f;
	pOut->y = (float)iG/255.0f;
	pOut->z = (float)iB/255.0f;
	pOut->w = (float)iA/255.0f;
}

Not very clear? This function is called for each pixel in our texture, with the address of that pixel passed as a 2D vector in ‘pTexCoord’. Values come in ‘pTexCoord’ as homogeneous coordinates, ranging from 0.0f to 1.0f. Where (0,0) is the upper-left corner of the texture, and (1,1) is the bottom-right. We are required to fill a 4D vector ‘pOut’ with the color value of the corresponding pixel in our map, and this is just what we do.
We send the address of our CSIBCPixMap object to this function so we can query it for pixel values. CSIBCPixMap::GetPixel() will return the color value of a specific pixel in the range 0-255 inclusive. However, we’re required to output the color value in 0.0f-1.0f range, so we divide each color channel value by 255 and pass it to D3DX.

Superb, now all what’s left is to make CMyD3DApplication::dotXSILoadMaterialLibrary() aware of those two shader nodes, so the code that checks for shader ProgIDs becomes: 

if (strstr(ppShaders[j]->GetProgID(),"DSK3DRenderer"))
	pNode = dotXSILoadDSK3DRenderer(ppShaders[j]);
else if (strstr(ppShaders[j]->GetProgID(),"DSK3DParamTexture"))
	pNode = dotXSILoadDSK3DParamTexture(ppShaders[j]);
else if (strstr(ppShaders[j]->GetProgID(),"DSK3DTextureStage"))
	pNode = dotXSILoadDSK3DTextureStage(ppShaders[j]);
else continue;

if (pNode == NULL)
{
	DXTRACE("Failed to load shader %s\n",ppShaders[j]->Name().GetText());
	continue;
}
Voila! Now our application can read the three shaders DSK3DRenderer, DSK3DParamTexture and DSK3DTextureStage.


Como Estais Amigos

Whew! That was a lot of info we covered today.
We used the Semantic Layer, User Data Fields, .pic textures, built a real-time shaders system, opened DSK|ShaderBass® nodes and handled texture coordinates.

Yes, we still have a lot of things to cover, animation and model hierarchy is a must, then we can delve into envelopes and character animation.
The code I presented here is somewhat similar to what DirectSkeleton® does when reading DSK|ShaderBass® nodes. However, DirectSkeleton handles many other aspects including parameter animation.

The code presented here is not ideal! It’s there for simplicity. You can make much better code by forward planning with your engine’s capabilities in mind.
I also omitted some more advanced information about User Data, which talks about User Data release functions. Maybe we’ll use that in our next trip.

By now, you should have a clear understanding of how the two FTK APIs interact with each other, so you can decide which one suits you better. And remember, you can always switch to the I/O Layer from within the Semantic Layer.

Did you notice how our programming paradigm changed between the I/O version and SL version?
In the older one, we were placed in a situation like:
"Oh! We have a mesh here? Load it. What’s next? Oh darn! That’s an SI_Envelope, so that mesh was enveloped! fix it …etc".
While in the Semantic Layer things go like: "Let’s check out all meshes in the scene, and for each mesh, is it enveloped? What material it takes?".
So things changed from trying to handle the unexpected into loading what’s expected, which is much less error prone.

The zip file attached with this paper contains two sample applications. The first is the same application we built in the previous tutorial, but done with the Semantic Layer. The second application is the one capable of opening DSK|ShaderBass® real-time shaders.
The projects were done on Visual Studio.NET® 2003, but should compile safely under other IDEs, as I avoided compiler-specific features.

That’s it! This is where I’ll stop screwing up your day with this tutorial. Until the next time, bye!


See Also

Built on Saturday, Sep 27, 2003