It is currently Mon Nov 19, 2018 3:13 pm

All times are UTC




Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 100 posts ]  Go to page Previous  1 ... 3, 4, 5, 6, 7  Next
Author Message
 Post subject:
PostPosted: Thu Feb 25, 2010 8:29 pm 
Offline

Joined: Fri Feb 19, 2010 5:04 pm
Posts: 115
Location: England
Tiran wrote:
Before I reply to the other replies to my posting I like to lay common grounds. What do we have to restrict.

Python code = any code written in Python and included with MOUL as part of the Python standard library or age code.

There may be a distinction between code which is part of the game engine and code which is part of an age.

It's feasible to rely upon vetting for code which users have to explicitly choose to download. But I'm envisaging an ecosystem where ages are downloaded on demand the first time you use the age's linking book. The age might even be running on a different server to the age containing the linking book. In that situation, the interpreter has to be bulletproof against bugs in age scripts.

If it was JavaScript, I'd use a separate interpreter (JSRuntime) for the engine and for the age. But I don't know how feasible it is to do that in Python. In the worst case, you could have a completely separate process for running age scripts, but that has costs in terms of both effort and efficiency.

Tiran wrote:
* Python code must not read or write any files from the file system

It must not have unrestricted access to the file system. It may be acceptable for an age to read or write some files. E.g. you could give each age its own subdirectory within the "Uru Live" directory, and only allow it to read or write files in that directory. It would need to be prevented from writing EXE or DLL files; in fact, it would probably have to be limited to a fixed set of safe types (which includes restricting the format as well as the extension; Windows sometimes ignores the extension and auto-detects the type based upon content).

Tiran wrote:
* Python code must not be able to open sockets in order to make network connections to different hosts

Right. It should be able to communicate with the server running the age (same-origin policy), but this would be via the game protocol; it wouldn't be able to use "bare" sockets. This would probably be necessary regardless of any security issues, as servers will typically need some kind of login, but age scripts shouldn't have access to the player's login and password (they may have a valid use for the avatar name, but not the login).

Tiran wrote:
* Python code must not be able to load extensions (DLLs) which might allow users to add new features
* Python code must not be allowed to load additional modules from the file system

Like with files, it might be reasonable to allow a fixed set of extensions. IOW, you might not want the entire range of available functionality to be visible without explicit "import"s. But we would probably want to limit extensions anyhow, so you don't run into a mess of "age X needs extension Y" dependencies.


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Thu Feb 25, 2010 9:42 pm 
Offline

Joined: Tue May 09, 2006 4:24 pm
Posts: 3120
Location: Aachen, Germany
Keikoku wrote:
There may be a distinction between code which is part of the game engine and code which is part of an age.

It's feasible to rely upon vetting for code which users have to explicitly choose to download. But I'm envisaging an ecosystem where ages are downloaded on demand the first time you use the age's linking book. The age might even be running on a different server to the age containing the linking book. In that situation, the interpreter has to be bulletproof against bugs in age scripts.

If it was JavaScript, I'd use a separate interpreter (JSRuntime) for the engine and for the age. But I don't know how feasible it is to do that in Python. In the worst case, you could have a completely separate process for running age scripts, but that has costs in terms of both effort and efficiency.


How familiar are you with way MOUL works? The Python part is mostly used for three purposes: KI including the chat, translation and wiring the interactive parts of an age. The game exposes a set of functions and objects to Python in order to interact with the server. The functions and objects are implemented in C or C++ and provided over the C API of Python.

The compiled Python code and age settings are downloaded from the server as encrypted data everytime you log into the game. The game takes so long to load because all data is downloaded and non is cached. It's not an ideal situation but it makes it impossible to read or tamper with the code. This won't change to prevent users from inserting their own code into the game.

Yes, Python supports isolated subinterpreters that may be used to isolate official code from user contributed code. Subinterpreters complicate the interface between game code and Python code. I recommend against them for the time being.

Keikoku wrote:
It must not have unrestricted access to the file system. It may be acceptable for an age to read or write some files. E.g. you could give each age its own subdirectory within the "Uru Live" directory, and only allow it to read or write files in that directory. It would need to be prevented from writing EXE or DLL files; in fact, it would probably have to be limited to a fixed set of safe types (which includes restricting the format as well as the extension; Windows sometimes ignores the extension and auto-detects the type based upon content).


Any write access may be abused. All user data should be stored on the server, too. A local settings file or save games isn't a good idea for an online game. The game could provide a set of function to store local data for stuff like Jalak setups. Direct read/write access is definitely a no-go.

Keikoku wrote:
Like with files, it might be reasonable to allow a fixed set of extensions. IOW, you might not want the entire range of available functionality to be visible without explicit "import"s. But we would probably want to limit extensions anyhow, so you don't run into a mess of "age X needs extension Y" dependencies.


No, a user must not be able to inject Python code. As soon as you allow to execute arbitrary Python code a user can hack the game. You don't want spam bots, Fly Mode, UserKI or other stuff on the official servers.

_________________
Image
[KI again #01792364]| Uru images | KI guide


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Thu Feb 25, 2010 10:00 pm 
Offline

Joined: Tue May 09, 2006 12:33 am
Posts: 1182
Location: British Columbia, Canada
Also note that each plPythonFileMod object runs in its own instance of the Python interpreter.


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Fri Feb 26, 2010 3:26 am 
Offline

Joined: Fri Feb 19, 2010 5:04 pm
Posts: 115
Location: England
Tiran wrote:
The compiled Python code and age settings are downloaded from the server as encrypted data everytime you log into the game. The game takes so long to load because all data is downloaded and non is cached. It's not an ideal situation but it makes it impossible to read or tamper with the code. This won't change to prevent users from inserting their own code into the game.

Once the client source is released, you can't prevent users from modifying it. If you want to protect the server, you have to do so on the server, not the client.

Tiran wrote:
No, a user must not be able to inject Python code. As soon as you allow to execute arbitrary Python code a user can hack the game.

I'm not talking about arbitrary Python code, I'm talking about standard libraries which scripts can use. There are reasons why you might not want to do this (e.g. compatibility issues), but security isn't one of them.

Tiran wrote:
You don't want spam bots, Fly Mode, UserKI or other stuff on the official servers.

If Uru is to be open-source, you can't prevent people from playing with modified clients. It's hard enough to do that with a closed client; witness the amount of effort Blizzard puts into anti-cheating mechanisms.


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Fri Feb 26, 2010 1:08 pm 
Offline

Joined: Tue May 09, 2006 4:24 pm
Posts: 3120
Location: Aachen, Germany
Keikoku wrote:
Once the client source is released, you can't prevent users from modifying it. If you want to protect the server, you have to do so on the server, not the client.


If we would life in a perfect world you'd be right. With online games such as MOUL the server has to trust the client partially. It's not possible to verify every movement on the server. The lag would make the game sluggish and the load would kill the server. I predict that each game server is going to trust a known and verified set of files (ages, Python scripts and MOUL client) only.

Keikoku wrote:
I'm not talking about arbitrary Python code, I'm talking about standard libraries which scripts can use. There are reasons why you might not want to do this (e.g. compatibility issues), but security isn't one of them.


By security I'm talking about two sides: security of the user's computer and the integrity of the server. Once you the user to execute code he may abuse the code to circumvent a puzzle or jump higher than others. The standard library of Python is designed with a different goal in mind.

Keikoku wrote:
If Uru is to be open-source, you can't prevent people from playing with modified clients. It's hard enough to do that with a closed client; witness the amount of effort Blizzard puts into anti-cheating mechanisms.


An open source client and server doesn't necessarily mean you can connect to a server with any client.

_________________
Image
[KI again #01792364]| Uru images | KI guide


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Fri Feb 26, 2010 4:13 pm 
Offline
Creative Kingdoms

Joined: Tue May 09, 2006 8:06 pm
Posts: 6223
Location: Everywhere, all at once
Welcome back Tiran. Your presence has been missed (well, I did anyway) and the community feels a little more complete with you around looking after things..

_________________
OpenUru.org: An Uru Project Resource Site : Twitter : Make a commitment.
Image


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Fri Feb 26, 2010 4:54 pm 
Offline

Joined: Tue May 09, 2006 4:24 pm
Posts: 3120
Location: Aachen, Germany
Thanks JW! I've to defend my place in the top ten posters list somehow ... :) You and Rusty are still unbeatable but I may be able to get past TomahnaGuy and D'Lanor in a few weeks.

_________________
Image
[KI again #01792364]| Uru images | KI guide


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Fri Feb 26, 2010 10:25 pm 
Offline

Joined: Fri Feb 19, 2010 5:04 pm
Posts: 115
Location: England
Tiran wrote:
Keikoku wrote:
Once the client source is released, you can't prevent users from modifying it. If you want to protect the server, you have to do so on the server, not the client.

If we would life in a perfect world you'd be right. With online games such as MOUL the server has to trust the client partially. It's not possible to verify every movement on the server. The lag would make the game sluggish and the load would kill the server. I predict that each game server is going to trust a known and verified set of files (ages, Python scripts and MOUL client) only.

The server knows nothing about the client beyond what the client chooses to tell it. It has no way of knowing that a modified client is being used unless the client chooses to provide this information. If servers start modifying their behaviour (e.g. refusing access) according to the client identifier provided by the client, clients will simply allow the user to choose how the client identifies itself (in the same way that many web browsers have an option to masquerade as Internet Explorer).

It has nothing to do with being in a "perfect world", and everything to do with being in the "real world". In computing (or in most things, really), what you want only matters insofar as it intersects with what's possible.

Tiran wrote:
Keikoku wrote:
I'm not talking about arbitrary Python code, I'm talking about standard libraries which scripts can use. There are reasons why you might not want to do this (e.g. compatibility issues), but security isn't one of them.

By security I'm talking about two sides: security of the user's computer and the integrity of the server. Once you the user to execute code he may abuse the code to circumvent a puzzle or jump higher than others. The standard library of Python is designed with a different goal in mind.

The two sides are symmetrical. The client can (or at least should be able to) choose what code it wishes to run. Similarly, the server can choose what code it wishes to run. The server cannot limit which code the client chooses to run, or vice versa.

The desirability of restricting clients is irrelevant; it simply isn't possible.

You can't even limit access to closed-source versions of a client if they are based upon open source. When you have source for a nearly-identical client, it's too easy to reverse-engineer the closed-source version and figure out how to mimic it. Encrypted binary? That's how copy-protection mechanisms (Safedisc, SecuROM, etc) work, and it doesn't seem to provide much of an obstacle to the pirates.

Tiran wrote:
An open source client and server doesn't necessarily mean you can connect to a server with any client.

Actually, yes it does. You can make all sorts of rules about which clients are permitted, but ultimately you have no way to enforce such rules when the server is unable to tell which client is being used.

If you want to prevent cheating, you have to monitor the client's behaviour, e.g. if a client makes impossible claims about the player's movement (jumping too high). IOW, you cannot detect whether the client has an option to cheat, only whether the player makes use of that option.

But, really, who is going to go to the trouble of cheating? And what does it matter if they do? No-one's handing out prizes, and I can't really foresee gold-farming taking off. And MOUL isn't the kind of game where you can easily "grief" people. There's no loot to steal, no way to kill someone, no way to get into someone else's age instance without an invite; you can't even block someone's path.

For now, I'd rather focus on what's both important and achievable, i.e. preventing execution of malicious (or even accidentally dangerous) code. For all I know, the execution environment may already be well secured. But even if it is, I'm fairly sure that people are going to want to extend the engine, in which case our job is to figure out how to do so in a secure manner.


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Sat Feb 27, 2010 8:11 pm 
Offline
Obduction Backer

Joined: Wed May 31, 2006 4:09 am
Posts: 20
It seems to me that it is infeasible for the server to check the validity of everything coming from the client, and it is impossible to prevent the client from sending whatever it wants. This is the same problem that any other online game has. Of course, it will be easier to create malicious messages if the game is open source, but it's not a new problem by any means. Most people deal with it by banning the malicious clients.

As I understand it, in Until Uru, clients had to authenticate with Cyan's server before they could connect to other servers. You could do something similar here, and have some global ban list for accounts on the auth server. I'm not saying that Cyan has to run it, or even that all servers have to use it. You could have multiple auth servers, each with their own cluster of game servers that use them. The important thing is, malicious clients can be stopped at a higher level than just per-server.

There is no perfect solution to this, though.

Tiran wrote:
The compiled Python code and age settings are downloaded from the server as encrypted data everytime you log into the game. The game takes so long to load because all data is downloaded and non is cached. It's not an ideal situation but it makes it impossible to read or tamper with the code. This won't change to prevent users from inserting their own code into the game.


I'll agree with Keikoku here; once Uru is open source it will be trivially easy to have the client run whatever code it wants to, regardless of what the server tells us. You can probably even manage this with a closed client by intercepting the server messages and replacing them with your own code.

However, as I said above (in agreement with Tiran) it is just not possible to check all client messages for validity. I think what we should do instead is try to mitigate any permanent damage to the server. I don't really care if I see an avatar flying around the city (at least, I won't care for that long), but I do care if the flying avatar manages to corrupt the server's saved data.

Equally, I don't really care if people cheat to get ahead in Uru. It's not like their having solved all the ages makes gameplay less fun for me. We just have to make sure no matter what one client says, it doesn't crash the server, damage it, or slow it down like crazy. We don't have to ensure that all clients make sense.

(Once it becomes clear a player is flying around/cheating/whatever, you can ban him. As long as there is no permanent damage to the server, it's only a temporary problem.)

Paradox wrote:
Also note that each plPythonFileMod object runs in its own instance of the Python interpreter.


This is very interesting, and it should simplify any sort of selective security if we have to go down that route.


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Sat Feb 27, 2010 9:29 pm 
Offline

Joined: Fri Feb 19, 2010 5:04 pm
Posts: 115
Location: England
agrif wrote:
It seems to me that it is infeasible for the server to check the validity of everything coming from the client, and it is impossible to prevent the client from sending whatever it wants. This is the same problem that any other online game has. Of course, it will be easier to create malicious messages if the game is open source, but it's not a new problem by any means. Most people deal with it by banning the malicious clients.

Commercial products have a number of advantages here.

First, the client is closed-source, which makes it harder for people to figure out how to hack it. But not impossible; copy-protection code isn't merely closed but deliberately obfuscated, yet people figure out how to identify and disable it within days (or even hours) of a product being released.

Second, in the event that the server operators detect prohibited conduct, they can terminate the account. This actually means something when getting another account means buying another copy of the game or purchasing play-time. But if all you need to get a new account is a new email address, banning an account isn't a particularly effective weapon.

Third, commercial MMO operators can typically afford to send lawyers after people distributing unauthorised clients, patches, anti-anti-cheating tools, etc. Actually, for an open-source project, it's not just a question of being able to afford lawyers but of the law itself. While individual server operators can prohibit the use of certain tools, mods, hacks, etc on their server, they can't prevent the distribution of tools which might be allowed elsewhere.


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Sat Feb 27, 2010 9:53 pm 
Offline
Obduction Backer

Joined: Wed May 31, 2006 4:09 am
Posts: 20
Keikoku wrote:
Second, in the event that the server operators detect prohibited conduct, they can terminate the account. This actually means something when getting another account means buying another copy of the game or purchasing play-time. But if all you need to get a new account is a new email address, banning an account isn't a particularly effective weapon.


I meant to add that account banning should be combined with IP banning and anything else we can think of. This should be combined with some sort of appeal system (so that banned dynamic IPs don't keep legitimate people from playing) and a way to keep bots from creating a mass amount of accounts (like a captcha). This is still not the best system, but it should at least inhibit hacking.

I can't see hacking being a pervasive problem with a system like this. Sure, persistent hackers can always continue, but this is true of many systems. I hardly ever see so much griefing in, say, the Ubuntu IRC channel that it becomes unusable.


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Sun Feb 28, 2010 7:09 am 
Offline
Obduction Backer

Joined: Thu Jun 08, 2006 7:01 pm
Posts: 1890
Of course you want to harden the server against Vault corruption. (I expect that this is already done, since MOUL ran for over a year with no wipes.) But you also want to protect against illegal puzzle states, getting into closed areas, using linking books that should not yet be visible, etc.

This didn't really come up in MOUL because public areas didn't have puzzles; if you cheat in a private instance, so what. (And Cyan wouldn't release an Aegura update with a new linking book until a few days before it was scheduled to "really" appear.) But it's worth thinking about in the long term.

Somewhere back in ancient days I sketched out a scheme where the server would validate all *state-changing* message from every client. This is not trivial, because "state-changing" includes (for example) moving from outside the Hood book room to inside it. (If the doors are locked, this is an invalid state change; we rely on that protection to keep people away from a private Delin instance, for example.)

So you have to divide up the Age into a (small) number of areas, and keep track of which area each player is in. It's some extra server load, but not a lot, because players cross area boundaries rarely.

With this model, a client can implement flymode and fly into the book room -- but then none of the books will work. The server rejects the area change, so it thinks you're still outside. (Similarly for flying around Teledahn and pulling levers, jumping on the bridge, etc, etc.)

This guarantees that the server state remains consistent even with respect to puzzle state.

Down side: opens up more room for client/server sync bugs. If the server loses a movement message, you might find linking books and levers broken for no good reason. I think this is manageable, with a combination of redundancy and feedback.

(It also doesn't cover using flymode to go somewhere valid *fast*. E.g., it doesn't prevent a player from using a hacked client to solve Delin/Tsogal solo. I can live with that.)

_________________
Andrew Plotkin -- Seltani founding member


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Fri Mar 12, 2010 5:37 am 
Offline
Obduction Backer

Joined: Mon Jan 15, 2007 6:03 pm
Posts: 96
It would be nice to prevent people in common areas from flying around and such. Otherwise it might become a common occurrence in the city to see people flying, which would kind of ruin it. As people have said, what people do in their own ages is their own business.

But I suppose it's hard to validate if the client has control over all of the avatar's movements.

With regards to malicious Python scripts from the server -- not sure if this has been stated explicitly already, but it is generally not possible (at least in CPython, the standard Python implementation which I assume Cyan is using) to secure Python against malicious scripts. Even if you want a game which only lets Python scripts call functions within the game engine, they can still open networks, files, and anything else. Several attempts to secure Python, such as the deprecated rexec library, have been found to have unfixable holes. (I spent the last two years writing a web app which lets students run Python code on our servers -- we solve this problem with a full chroot jail per user.)

The only way to guard against malicious Python scripts is to hand-inspect them before letting them go out to clients. Incidentally, this is why games use Lua, not Python, because that language *is* restricted by default to only the game-supplied function calls.


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Fri Mar 12, 2010 1:19 pm 
Offline

Joined: Fri Feb 19, 2010 5:04 pm
Posts: 115
Location: England
Eat_My_Shortz wrote:
It would be nice to prevent people in common areas from flying around and such. Otherwise it might become a common occurrence in the city to see people flying, which would kind of ruin it. As people have said, what people do in their own ages is their own business.

But I suppose it's hard to validate if the client has control over all of the avatar's movements.

In games where cheating is an issue, the server is the final arbiter of the game state, including movement. But that's extra work for the server, so it's not something you'd want to do unless there was a clear need. You're probably better off just relying upon server administrators to monitor users and ban those who don't behave. That also deals with forms of misbehaviour which can't be detected algorithmically (e.g. harassment via text or voice chat).

Quote:
With regards to malicious Python scripts from the server -- not sure if this has been stated explicitly already, but it is generally not possible (at least in CPython, the standard Python implementation which I assume Cyan is using) to secure Python against malicious scripts. Even if you want a game which only lets Python scripts call functions within the game engine, they can still open networks, files, and anything else. Several attempts to secure Python, such as the deprecated rexec library, have been found to have unfixable holes.

MOUL uses a custom interpreter, so the situation is somewhat easier than trying to harden the stock Python interpreter. It's still harder than hardening a "minimalist" language such as Lua or JavaScript, though (mainly due to Python's habit of exposing its internals). If and when source code is available, we'll have a better idea of whether the Python implementation can be hardened sufficiently; until then, it's just guesswork.

Quote:
The only way to guard against malicious Python scripts is to hand-inspect them before letting them go out to clients.

Which would largely defeat the point of open-source.

Quote:
Incidentally, this is why games use Lua, not Python, because that language *is* restricted by default to only the game-supplied function calls.

With a custom Python interpreter, you can control what is and isn't accessible from Python. The hard part is if you want a specific function to be accessible to some Python code but not universally accessible. Where all of the restricted-execution systems tend to fall down is that there's a vast number of places to "hang" code, which makes it almost impossible to enforce a distinction between different types of code (although it has been said that MOUL does use separate sub-interpreters for different bits of code, which is more robust than trying to isolate sections of code within a single interpreter).


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Sat Mar 13, 2010 11:36 am 
Offline
Obduction Backer

Joined: Mon Jan 15, 2007 6:03 pm
Posts: 96
Quote:
MOUL uses a custom interpreter, so the situation is somewhat easier than trying to harden the stock Python interpreter.

What? Cyan wrote their own Python interpreter?? Surely this isn't true... it would take a huge effort to write a working Python implementation (I know because a friend of mine has done it).
Quote:
Quote:
The only way to guard against malicious Python scripts is to hand-inspect them before letting them go out to clients.

Which would largely defeat the point of open-source.

No it wouldn't... you would still be able to see the source code, modify it for your own purposes, if you have a useful patch send it upstream so it can be used by servers and clients. This would apply to the game code as well as custom ages. It would then be up to the server to choose whether to support your particular custom Age, just as it's up to them whether to support your particular custom code.

In Ubuntu, it isn't the case that just anybody can get code in the official repositories (becoming part of the operating system). There is a strict vetting process. That doesn't mean that Ubuntu "largely defeats the point of open source."

Anyway I don't know about this custom interpreter business. I guess we'll see when the source is released.[/quote]


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 100 posts ]  Go to page Previous  1 ... 3, 4, 5, 6, 7  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to: