Difference between revisions of "LuaJIT"

From X-Plane SDK
Jump to: navigation, search
(Created page with "[http://luajit.org/ LuaJIT 2.0] is a just-in-time compiler for the Lua scripting language; it is the preferred way to embed Lua in a plugin. This technote describes the integrat...")
 
Line 9: Line 9:
 
= LuaJIT 2.0 and X-Plane =
 
= LuaJIT 2.0 and X-Plane =
  
LuaJIT 2.0 requires that Mac OS X 64-bit apps have their base addresses customized, as the default OS X memory map sets the entire bottom 4 GB of address space to zero page.  X-Plane 10.20 beta 3 and later fix this, providing at least 1.3 GB of address space to LuaJIT 2.0 (of the 1.8 GB maximum the custom LuaJIT allocator could use).
+
LuaJIT 2.0 requires that Mac OS X 64-bit apps have their base addresses customized, as the default OS X memory map sets the entire bottom 4 GB of address space to zero page.  X-Plane 10.20 has this fix applied, providing at least 1.3 GB of address space to LuaJIT 2.0 (of the 1.8 GB maximum the custom LuaJIT allocator could use).
  
 
'''The problem:''' during normal operation, OS X's system allocator and OpenGL stack will use free memory in the bottom 2 GB, potentially exhausting it under high scenery engine load.  Plugins utilizing Lua that are loaded late (e.g. in an airplane load) or need more memory mid-run may fail allocation since this critical "bottom 2 GB" of address space is taken.
 
'''The problem:''' during normal operation, OS X's system allocator and OpenGL stack will use free memory in the bottom 2 GB, potentially exhausting it under high scenery engine load.  Plugins utilizing Lua that are loaded late (e.g. in an airplane load) or need more memory mid-run may fail allocation since this critical "bottom 2 GB" of address space is taken.
  
= Temporary Development During 10.20 Betas =
+
X-Plane 64-bit for OS X solves this problem by pre-grabbing the entire LuaJIT address space for itself on startup and then providing this address space to Lua-based plugins on demand.  A special set of messages can be sent from plugins to X-Plane to allocate and release memory.
  
If you are working on porting a LuaJIT-based plugin to 64-bits during the 10.20 beta, test under low memory use conditions. To do this:
+
= Sample Code to Use X-Plane's LuaJIT Allocator =
* Run in an all-water area; don't load scenery.
+
* Turn texture resolution way down.
+
* Don't move the plane between locations a lot without restarting.
+
These techniques should help minimize pressure on Lua memory.
+
  
= A Real Solution =
+
<sample_code>
  
Laminar Research is working on a 'real' solution, e.g. one that reserves the bottom 2 GB of address space for LuaJIT even while the scenery engine is runningContact ben at x-plane dot com for more information. This tech note will be updated with the recommended practice once it is finalized.
+
/* Include XPLM headers, etc. */
 +
 
 +
struct lua_alloc_request_t {
 +
void * ud;
 +
void * ptr;
 +
size_t osize;
 +
size_t nsize;
 +
};
 +
 
 +
#define ALLOC_OPEN 0x00A110C1
 +
#define ALLOC_REALLOC 0x00A110C2
 +
#define ALLOC_CLOSE 0x00A110C3
 +
 
 +
/* lua interpreter */
 +
lua_State* l;
 +
 
 +
static void *lj_alloc_create(void)
 +
{
 +
struct lua_alloc_request_t r = { 0 };
 +
XPLMSendMessageToPlugin(XPLM_PLUGIN_XPLANE, ALLOC_OPEN,&r);
 +
return r.ud;
 +
}
 +
 
 +
static void lj_alloc_destroy(void *msp)
 +
{
 +
struct lua_alloc_request_t r = { 0 };
 +
r.ud = msp;
 +
XPLMSendMessageToPlugin(XPLM_PLUGIN_XPLANE, ALLOC_CLOSE,&r);
 +
}
 +
 
 +
static void *lj_alloc_f(void *msp, void *ptr, size_t osize, size_t nsize)
 +
{
 +
struct lua_alloc_request_t r = { 0 };
 +
r.ud = msp;
 +
r.ptr = ptr;
 +
r.osize = osize;
 +
r.nsize = nsize;
 +
XPLMSendMessageToPlugin(XPLM_PLUGIN_XPLANE, ALLOC_REALLOC,&r);
 +
return r.ptr;
 +
}
 +
 
 +
/* when you need to init LUA... */
 +
 
 +
  void *ud = lj_alloc_create();
 +
  if (ud == NULL)
 +
  {
 +
    /* handle error */
 +
  }
 +
  l = lua_newstate(lj_alloc_f, ud);
 +
 
 +
 
 +
</sample_code>
  
 
[[Category:Tech Notes]]
 
[[Category:Tech Notes]]

Revision as of 03:24, 8 December 2012

LuaJIT 2.0 is a just-in-time compiler for the Lua scripting language; it is the preferred way to embed Lua in a plugin. This technote describes the integration issues between X-Plane 10.20b3 OS X/64-bit and LuaJIT. These problems are specific to OS X and 64-bit; 32-bit plugins and Windows/Linux 64-bit plugins are not affected.

LuaJIT 2.0 and Address Space

LuaJIT 2.0 requires that all allocations used by JIT code be in the bottom 2 GB of address space. (This requirement comes from the use of signed 32-bit relative addressing for constants in the generated code.) To meet this requirement, LuaJIT has two modifications from a 'standard' Lua run-time:

  • The built-in Lua allocator attempts to grab memory from the lower 2 GB of address space using various OS-specific techniques. The allocator is a copy of dl-malloc sitting on top of this custom bottom-2-gb scheme.
  • lua_newstate is inoperative in the 64-bit version of LuaJIT 2.0. This stops client code from providing a custom allocator that ignores the < 2 GB rule (which would cause JIT-code crashes).

LuaJIT 2.0 and X-Plane

LuaJIT 2.0 requires that Mac OS X 64-bit apps have their base addresses customized, as the default OS X memory map sets the entire bottom 4 GB of address space to zero page. X-Plane 10.20 has this fix applied, providing at least 1.3 GB of address space to LuaJIT 2.0 (of the 1.8 GB maximum the custom LuaJIT allocator could use).

The problem: during normal operation, OS X's system allocator and OpenGL stack will use free memory in the bottom 2 GB, potentially exhausting it under high scenery engine load. Plugins utilizing Lua that are loaded late (e.g. in an airplane load) or need more memory mid-run may fail allocation since this critical "bottom 2 GB" of address space is taken.

X-Plane 64-bit for OS X solves this problem by pre-grabbing the entire LuaJIT address space for itself on startup and then providing this address space to Lua-based plugins on demand. A special set of messages can be sent from plugins to X-Plane to allocate and release memory.

Sample Code to Use X-Plane's LuaJIT Allocator


/* Include XPLM headers, etc. */

struct lua_alloc_request_t {
			void *	ud;
			void *	ptr;
			size_t	osize;
			size_t	nsize;
};

#define		ALLOC_OPEN		0x00A110C1
#define		ALLOC_REALLOC	0x00A110C2
#define		ALLOC_CLOSE		0x00A110C3

/* lua interpreter */
lua_State* l;

static void *lj_alloc_create(void)
{
	struct lua_alloc_request_t r = { 0 };
	XPLMSendMessageToPlugin(XPLM_PLUGIN_XPLANE, ALLOC_OPEN,&r);
	return r.ud;	
}

static void  lj_alloc_destroy(void *msp)
{
	struct lua_alloc_request_t r = { 0 };
	r.ud = msp;
	XPLMSendMessageToPlugin(XPLM_PLUGIN_XPLANE, ALLOC_CLOSE,&r);
}

static void *lj_alloc_f(void *msp, void *ptr, size_t osize, size_t nsize)
{
	struct lua_alloc_request_t r = { 0 };
	r.ud = msp;
	r.ptr = ptr;
	r.osize = osize;
	r.nsize = nsize;
	XPLMSendMessageToPlugin(XPLM_PLUGIN_XPLANE, ALLOC_REALLOC,&r);
	return r.ptr;
}

/* when you need to init LUA... */

  void *ud = lj_alloc_create();
  if (ud == NULL)
  {
     /* handle error */
  }
  l = lua_newstate(lj_alloc_f, ud);