Breaking Adobe DRM — Tips, Tricks, Workarounds

In iPhone: Reading Adobe/EPUB eBooks I gave instructions on how to move Adobe e-books, which are normally DRM-encrypted, to an iPhone for reading in an "open" e-book format. Digital rights management: it's "copy protection" that locks an e-book into a particular handheld reading device or a particular piece of software on a Windows PC or a Mac OS X computer. Many people who have paid for an e-book prefer to read it outside its prescribed environment. Hence, posts like this one.

For example, those who have paid for an e-book encrypted with Adobe's DRM scheme may want to read it on a iPhone, and iPad, or an iPod Touch. That can't happen unless the DRM encryption is removed.

Caution: Removing DRM encryption is illegal! You might think that, as long as you do not share the decrypted e-book with unauthorized users, it should be legally permissible to de-DRM a legally owned copy of an e-book.

Alas, it is not. In the United States, according to Wikipedia, per the Digital Millenium Copyright Act of 1998 " ... circumventing an access control, whether or not there is actual infringement of copyright itself" is illegal. DRM ("digital rights management") amounts to "access control": who may gain access to what copyrighted stuff, and when and how. DRM is also copy protection. DRM does both.

So when we strip DRM in order to move an e-book to an iPhone, we are "circumventing an access control."

Actually, this and other blog posts of mine are likewise in violation of DMCA, in that they tell people how to strip DRM and provide links to software that does that. I guess I'll be seeing you all in the old proverbial cellblock someday ...

Anyway ... taken from the long series of comments people have left below that original post, here are some tips, tricks, and workarounds that may help those who run into trouble in removing DRM encryption from Adobe e-books.


The Unzip/Rezip Trick

Some users of the ineptepub Python script see ...

Error: problem decrypting session key

... when they shouldn't. This error message is a general one that basically means ineptepub couldn't decrypt the e-book, for any of a variety of reasons. One possible reason is that the internal format of the e-book is faulty.

Some ineptepub users get a more-specific message like:

Error: File name in directory "OEBPS/BookTitle_chap-nn ... "

This error is typically caused by the same problem with the e-book and has the same solution, and it likewise means that ineptepub has been unable to decrypt the e-book.

What is the underlying cause? The internal format of an Adobe e-book is the same as that of any standard EPUB: it's a ZIP file. (See this Wikipedia article for more on the "open" EPUB format.) The various folders and files that make up any EPUB e-book, including any EPUB e-book that uses Adobe DRM, will have been compressed into a ZIP format archive. (The Wikipedia article says that the "OEBPS Container Format (OCF) 1.0," defines the standard for how the e-book's contents, which are a set of folders that contain a bunch of files, are put in a single "container" file. The OCF container file "collects all files as a ZIP archive.")

Whenever the e-book is read, it first has to be expanded from its ZIP form. Normally, this is done by the reader app. ineptepub in fact tries to do the same thing, and when the internal format is not as expected, an error results.

This happens a lot on Adobe e-books downloaded by Adobe Reader on a Mac. It doesn't seem to affect Adobe e-books downloaded by Adobe Reader on a Windows PC.

A workaround that sometimes helps is to unzip the original Adobe e-book, rezip it again while using the correct internal format, and then use ineptepub with the rezipped copy as its input file.

That can be done in the Mac OS X environment in the following way. For purposes of this example, I'll assume that the name of the downloaded .epub file is The_Poisonwood_Bible.epub. Like most downloaded Adobe e-books, it is typically located in ~/Documents/Digital Editions/. (The first node, ~, refers to your username's Home directory. To open that directory in Finder, simply type Shift-Command-H.)

Here are the steps you will need to take after opening ~/Documents/Digital Editions/ in Finder.

1. Make a copy on the Desktop of The_Poisonwood_Bible.epub. (Working with a copy of the original file is a good idea because you don't want to take a chance on ruining the original.) The easiest way to copy the file to the Desktop is to drag it there from its original folder while holding down the Option key.
2. Change the filename extension on the copied version of The_Poisonwood_Bible.epub from .epub to .zip. To do that in Finder, click on the file itself as it sits on the Desktop, click on the name of the file to highlight it, double-click on the .epub extension, and use the keyboard to change that extension to .zip. Or, if the extension is not visible, select the file and type Command-I — or choose Get Info from Finder's File menu — and edit the extension as it appears in the Name & Extension text box in the Get Info window; you simply replace .epub with .zip in the text box. Then close the Get Info window.
3. If asked whether you really want to change .epub to .zip, select "Use .zip".
4. You should now have a file on your Desktop called The_Poisonwood_Bible.zip. It is exactly the same file as The_Poisonwood_Bible.epub, but with a different filename extension.
5. Double click The_Poisonwood_Bible.zip on the Desktop, so that Archive Utility on your Mac unzips it into a new folder, also on your Desktop. (If double clicking the .zip file doesn't start Archive Utility, you may have to right-click on the .zip file and, from the pop-up menu, choose Open With: Archive Utility. Or, if that option doesn't seem to be available in the pop-up menu, you can manually open Archive Utility from /System/Library/CoreServices/Archive Utility.app and choose Expand Library from the Archive Utility's File menu, then navigate to and select The_Poisonwood_Bible.zip on the Desktop in the standard file open dialog.)
6. You should now have a new folder on your Desktop called The_Poisonwood_Bible. (Check that it contains a file called mimetype and two folders, META-INF and OEBPS. However, the contents of this folder are not really of interest to you.)
7. Trash the The_Poisonwood_Bible.zip file on the Desktop, as you do not need the copy you made of your original e-book file any more.
8. Download and unzip ePub Zip 1.0.2.app.zip, if you don't already have it. Double click it to expand it to ePub Zip 1.0.2 on the Desktop. (See above for what to do if double-clicking does not automatically expand a ZIP file in Archive Utility.)
9. Drag the icon for the new The_Poisonwood_Bible folder to ePub Zip 1.0.2 and drop it onto the ePub Zip 1.0.2 icon. (Note: You must drag and drop your The_Poisonwood_Bible folder and not your The_Poisonwood_Bible.zip file, which should now be in the Trash anyway.) This drag-and-drop operation causes an AppleScript to run in the background. That AppleScript's function is to rezip your e-book the right way!
10. You should at this point see a dialog box, "The_Poisonwood_Bible.epub created successfully," and a The_Poisonwood_Bible.epub file should magically show up on your Desktop. This The_Poisonwood_Bible.epub file is your correctly rezipped e-book.
11.  Use the new The_Poisonwood_Bible.epub file as the input file to the ineptepub script.

If ineptepub still doesn't work, the problem lies elsewhere than with the internal ZIP format of the e-book.

If you use Windows, I imagine something like WinZip might offer the ability to do the same unzip/rezip trick. Sorry I can't give you detailed instructions about that. However, as I said earlier, this problem doesn't seem to affect Windows users very often, if at all.

Installing PyCrypto on Snow Leopard

The instructions for Mac OS X users that I gave in iPhone: Reading Adobe/EPUB eBooks are basically for users of the 10.5.x (" Leopard") version of that operating system. Many users are now on the 10.6.x ("Snow Leopard") version. Though my procedure basically works with Snow Leopard, some Snow Leopard users have had trouble with installing PyCrypto, the Python Cryptography Toolkit, in Step 3 of my original procedure.

The basic problem has to do with Snow Leopard's use of two Python versions. Version 2.5 is the one that you need to use to run the two Python scripts, ineptkey and ineptepub, that my instructions call for. ineptkey is the script that generates the adeptkey.der ADEPT key file, and as such does not require PyCrypto, but ineptepub, which is the script that actually decrypts the e-book, does need PyCrypto. PyCrypto is basically a set of library files Python needs to access when it runs ineptepub. This set of library files needs to be installed in the right place in your Mac's folder hierarchy. (When I refer generically to just plain ineptepub herein, I mean specifically the ineptepub_v2 applet that is created in Step 6 of my original instructions.)

Some users of Snow Leopard are finding that the PyCrypto installer is putting PyCrypto in

/Library/Frameworks/Python.frameworks/versions/2.6/lib/python2.6/site-packages/

That is the wrong place! The right place for it is

/Library/Python/2.5/site-packages/

Solution: after installing PyCrypto in Step 3 of the original procedure, simply copy all of the contents of the

/Library/Frameworks/Python.frameworks/versions/2.6/lib/python2.6/site-packages/

folder to

/Library/Python/2.5/site-packages/

In Finder, you can do this by highlighting all of the items contained in the Python 2.6 /site-packages/ folder and dragging them with the option key held down to the Python 2.5 /site-packages/ folder. (Be aware that you may have to create the latter folder manually in the /Library/Python/2.5/ folder.) This option-drag procedure makes a copy in the destination /site-packages/ folder of all of the highlighted items.

Among the items that need to be copied in this way are a folder called Crypto, along with all its contents, and a file called pycrypto-2.0.1-py2.5.egg-info. You should check to make sure that these items and (in the case of the Crypto folder) their contained items get properly copied. If the latter file is named pycrypto-2.0.1-py2.6.egg-info, change the name to pycrypto-2.0.1-py2.5.egg-info manually.

At that point, you will hopefully be able to proceed with Steps 4-7 of the original procedure.

This applies to a Mac, by the way, and does not apply to Windows computers.

Which Build Applet Version To Use?

Some Mac users have gotten to Step 6 and had trouble where I tell them to right-click the ineptepub_v2.py script file and choose Open With: Build Applet. This is intended to cause an application, Build Applet, that is included with Python on a Mac, to change the text file ineptepub_v2.py into an "applet" — a double-clickable application — called ineptepub_v2.

Accordingly, right-clicking on ineptepub_v2.py brings up a menu that has an Open With: item, and one of the sub-items of that item should be just plain Build Applet.

But some users double-click on the ineptepub_v2.py script file and see, for example, Open With: Build Applet 2.6.0 and Open With: Build Applet: 2.5.4!

How you handle this problem depends on whether you are on Snow Leopard or Leopard.

(1) If you are on Snow Leopard, just choose Open With: Build Applet: 2.5.n — i.e., choose the version of Build Applet that has 2.5 in it.

(2) If you are on Leopard, which natively supports only Python 2.5, you probably have at some point tried to install some version of Python 2.6. The two versions of Python, 2.5 and 2.6, are different enough that ineptkey and ineptepub won't work properly with Python 2.6 on a Mac, and yet it may be the case that the default version of Python that your Mac is trying to use is in fact 2.6.

Since Mac OS X 10.5.n normally doesn't come with any version of Python other than 2.5, your having at some point installed (or tried to install) 2.6 may gum up the works.

I recommend that you do a clean install of Mac OS X 10.5.n, using the original install DVD, and also re-installed the Xcode Developer tools, which is a package that contains Python 2.5, at the same time. This will eliminate any vestiges of whatever Python 2.6 installation currently inhabits your Mac. (See the next topic for how to do a clean install.)

On the other hand, you may feel you have some reason not to do a clean Mac OS X 10.5 install. If so, your best bet may just be to choose Open With: Build Applet: 2.5.n, just as if you were on Snow Leopard. The result may be a usable ineptepub_v2 applet. If so, fine. If not, a clean install may be your only recourse.

Doing a Clean Install of Mac OS X 10.5.n

I recommend doing a clean install of Mac OS X 10.5.n, if you are on Leopard and not Snow Leopard and you are having trouble with getting the procedure to work that I laid out in my original post, iPhone: Reading Adobe/EPUB eBooks. This of course applies to Mac users only, not users of Windows PCs.

Mac OS X 10.5.n is the version that I currently use on my Mac, and the instructions in my original post apply mainly to it. As far as I know, they also work for Snow Leopard users, except for the problem mentioned earlier in which PyCrypto gets installed into the wrong folder. Snow Leopard users who run into other problems with my procedure may want to consider doing their own clean install of Mac OS X 10.6.n, but I cannot advise them about that.

To do a clean install of Mac OS X 10.5.n:

Click on this link to an excerpt from Mac OS X: Missing Manual  — Leopard, by David Pogue. Scroll about halfway down the page to the The Clean Install ("Archive and Install") section. Follow the instructions there.

To install the Xcode Developer tools:

Using your OS X 10.5.n Install DVD, navigate to and then double-click Mac OS X Install DVD:Optional Installs:Xcode Tools:Xcode Tools.mpkg on that DVD. You will be led, by the installer program which opens, through the necessary steps to install the Developer Tools package.

(See "Complete Steps to Perform a Clean OS X Reinstall on Your MacBook Air" for a how-to on doing a clean install on a Mac that lacks a DVD drive.)

Once you have done the clean install, including the Xcode Developer tools, go back and right-click the ineptepub_v2.py script file and choose Open With: Build Applet, which now should be the only Build Applet choice available. The result should be a usable ineptepub_v2 applet.

(2) If you are on Snow Leopard, then there probably should be two versions of Build Applet on your Mac, just as there are two versions of Python! In this case, just choose Open With: Build Applet: 2.5.n. Again, the result should be a usable ineptepub_v2 applet.

Reading Your Adobe E-Books on an iPad in iBooks

My original post was about reading your Adobe e-books on an iPhone in a reader app like Stanza, but you can now read them on an iPad (or an iPhone or iPod Touch) using Apple's free iBooks app.

If the free iBooks app from Apple works the same on an iPad as it does on my iPhone — and I believe it does, except I'm not sure to what extent you'll be able to take advantage of the larger screen or the standard assortment of fonts with the "open" EPUB books you derive from Adobe DRM'ed e-books — you're definitely in luck.

If you follow the instructions in my original iPhone: Reading Adobe/EPUB eBooks post (and, if need be, the problem-solving workarounds detailed herein) then for each Adobe e-book you decrypt, you will wind up with an open EPUB e-book file that is no longer locked into the Adobe Reader world.

Add that file (it will have an .epub filename extension, whether or not that extension is visible in Finder) to your iTunes library. On a Mac, you can just drag the file's Finder icon to the iTunes icon in the Dock or to the Books icon on the left side of the main iTunes window. On Windows, you can use, I believe, similar shortcuts.

Once the file shows up in iTunes' Books section, you can, in iTunes, just sync your iDevice — your iPad, iPhone, or iPod touch, as long as that iDevice is running iOS 4.0, Apple's latest operating system for mobiles — and then you can read the book in the iBooks app on the iDevice!

I find that the Adobe-derived open EPUBs I have tried so far in iBooks work exactly the same as Apple-sanctioned e-books from the iBookstore.

Fixing Missing Cover Artwork on open EPUBs in iBooks

But I have found that some but not all of the e-books I have treated in the way just described do not show up as having a pretty cover, either in iTunes or in the iBooks app on the iDevice. Some have no cover at all. Here's how to fix that:

In your web browser, bring up Google and click on Google's Images tab at the upper left of the browser window. In the Google search box enter the name of your e-book book and, to make sure Google knows exactly what you're looking for, the author's name, both in quotation marks. For example:

"an echo in the bone" "diana gabaldon"

Then, click on Search Images. Most of the results you see will be images of the actual book's cover!

Click on any thumbnail image you like — the larger the dimensions in pixels of the actual image, the better. At the top of the page that comes up, click on See full size image. Then left-click on the image that comes up and select Copy Image. (This works on a Mac using Safari. If you are on Windows, or if you are using another browser, you may have to make adjustments.)

You now have a copy of the image on your Clipboard. In iTunes, click on Books on the left side of the main window, then select the book the cover goes with and choose the Get Info menu item. (Mac shortcut: Command-I.) Click on the Artwork tab, then click in the (typically empty) artwork box. Paste the Clipboard's contents there, and click OK.

In a few seconds you will see the new cover artwork show up in the main iTunes window. Now sync your iDevice in iTunes. Once you do that, the new cover artwork will show up on your iBooks library bookshelf!