I just wanted to get some input as to how the participants in RYPN are using Adobe Acrobat to produce PDF's for library and archive uses. Are you using default settings or doing everything as a custom scan? If you are doing custom scans for each page of a document, would you care to comment about the scan settings you prefer?

My own preferences on this have been to use 200 DPI which seems to provide a better printed image than 150, and to do text only pages as black and white while doing halftone photo pages in grayscale with custom settings for brightness and contrast (sampling occasionally as you scan). This of course results in a bigger file size than if you were just to use the black and white default settings (usually 150 DPI) but seems to result in a much better print of the document.

Anyone have any thoughts to offer on this?

Thanks in advance for your opinions,

PC

Disk space is cheap.

Repeat.

Disk space is cheap.

Scan at 200 dpi, color. Even a black and white page has shades of "white" tone in the paper, and the ink has tones. A color image will capture the type characters more fully. I have every expectation that in ten years all printers will be color (and inexpensive), and you will then be able to reprint your scans visually identical to the original documents.

If there are color photos, scan at 300 dpi.

Be careful in the choice of scanning software. Scanning directly to pdf raises technical questions. If you scan to a native scanner file format, and save that, then you can easily "print to pdf" and create alternate versions of the pdf for different purposes. For example, a low resolution pdf for sharing, and a high resolution pdf for archive.

I use Photoshop to scan in that has a few more tools to tweak things.
Color at 200 makes a page about 1 MB.
Text isn’t too bad at 200dpi, I generally do not use color when scanning black and white. Halftone images are a problem, even at 400 dpi I can see moray. I use 300 or even 400dpi grayscale for pictures. To trade off file size I then tweak contrast and use about an 8 on the JPG compression.
A 400 DPI grey set an 8 after using the level function to clip some of the “dirt” out of the whites will be about 1.5MB.
Don’t know how this will look when I post but here is color at 200 vs. grayscale at 400. Extra points for knowing the location.

Grand Central Terminal in midtown Manhattan?

You're a winner, late 1930's photo of Grand Central from my old Compton's Encyclopedia.

I can appreciate the point that Steve makes, it is not possible to add quality to a scan after the fact, but if you scan for the best quality to begin with you can always go to a less space consuming format for convenience in e-mailing, etc. at a later date. There various priorities in scanning printed materials, from those that need best archival quality scanning to make them suitable for reproduction or future publication, to the materials that have information value but do not require high quality presentation, like EMD Maintenance Instructions. In the case of the latter, a lower resolution might be suitable rather than using up a lot of time making conversions when an e-mailable version is needed.

PC

I think the samples posted make my point. The 200 dpi color looks more "real" and complete than the 400 dpi greyscale, and it's even a smaller file size.

TigerDirect has a 2TB disk for $80. That is 2,000,000 mb, or about 2 million pages at the quoted rate per page.

Make sure the scanning software you purchase is intelligent enough to scan multiple pages into the same file.

WHOA, WHOA, WHOA!!! FOLKS!!! The current discussion completely misses the boat!

Scanning it is pointless if you can't find it again.

Many of you are "spoiled" by the type of annotated PDF files you see on Google Books. The ones where you're looking at something obviously scanned, yet magically if you drag your mouse across it, you pick up plain text you can cut-n-paste into Microsoft Word. This is not magic. You have to make it happen.

The PDF format allows the image to be tagged. A tag calls out a rectangle in the document and says "This glob of gray is the word 'hello'." When you sweep your mouse across it, you are picking up those tags. You notice this DOESN'T work on Buzz's images posted above. They are not tagged in this manner.

How do you make the tags? Typically your PDF authoring tool does that for you (if it can). It uses OCR, which requires enormous amounts of processor power, and works best on a high quality scan.

How do you keep a sane file size after the OCR is complete? Aha, you're not the first to ask. Adobe Acrobat is way ahead of you, it has features to let you compress the document to any degree you like. The key is starting with the highest resolution possible to begin with, so Acrobat has the most raw data to work with.

Buzz is saying "Hey, lower resolution makes a smaller JPEG" -- maybe, but it does not make smaller PDF's.

I loaded both of his JPEGs into Adobe Acrobat, did OCR on them and then hit "Reduce file size."
His 400 DPI JPEG -->> was 240kb --> reduced to 18kb
His 200 DPI JPEG -->> was 92kb --> reduced to 33kb
That's right. The higher resolution JPEG actually resulted in a lower resolution PDF, because Acrobat had more raw data to work with. It also did a better job with the OCR. (emphasis mine)




As for the image quality, Buzz and Lex are getting excited about artifacts of the halftone. At 200 DPI, his scanner software caught the halftone and converted it into grayscale for him, and that looks "oh so pretty". At 400 dpi, it's actually scanning the halftone proper, which offends the eye. Wrong conclusion. If you have excess resolution, you can always sample down. That is exactly what Acrobat's "Reduce file size" does with this halftone. Remember, letting Acrobat convert the 400dpi halftone resulted in a smaller file in the end. Adobe knows what they're doing.
Left is 36kb Right is 20kb


You're looking at a screensnap of the the pages in Adobe Acrobat side by side. (The darkness of the text is a function of the original scan and the resolution I'm viewing it at here. Fix that in scanner settings.) With both, you can mouse across them and pick up the text words. Typing "commercial interests" into my PC's search engine and both documents are found because of PDF's annotations.

I scan everything at 600dpi and I let Adobe Acrobat shrink it. This gives Acrobat the most to work with so I get the best shrinkage. Also the OCR works better. I recently scanned a Form 1099 (5MB), redacted out my social, did the above, and voila - 48KB.

If you volunteer for a nonprofit, you can get Adobe Acrobat for next to nothing from http://techsoup.org/.

As for hard drives, keep in mind a $80 hard drive is a piece o junk. But that's OK. Hard drives have gotten so big and so cheap that they've become the new "backup tape media". Don't buy just one - buy several. I use 4 of them on a 7-port USB hub, all redundant to each other, plus online backup (but not for raw files).

Decide whether you are trying to archive the information or the look and feel of the document. Reduction as described above compresses the information, but will lose the stains, watermarks, etc. of historic documents. Are you trying to preserve the look and feel of the document, or just the text? Fonts too: OCR scanning interprets the text, but can not save the peculiar typefaces in historic documents.

BOTH! That's what we're talking about. The compression is adjustable. Turn it down or turn it off, to your fancy. Like you say, disk is cheap.

You can't go by PDFs you pulled off the Internet. Those are deliberately over-compressed for the internet. Some people still have dial-up modems or AT&T wireless. What Buzz and I did was S.O.P. I also did the worst-case scenario by reworking something already compressed, and recompressing it. That causes severe image damage. If Buzz had scanned that at 600dpi and sent me a RAW file, the quality would be QUITE different even after I compressed. Compression works great actually, when you do it once - last.

Just because webmasters overcompress doesn't mean you need to. In fact Acrobat encourages you to save an uncompressed copy.


In this mode, PDF shows the user what was scanned. Fullstop. Peculiar typefaces and all. If your document has 1000 "T"s, each with slightly different font types, waterstains, etc., each of them is displayed as a special and unique snowflake. (If that offends your understanding of PDF, you are thinking of the other mode.) The OCR is nothing but a second "layer" of annotation, which is not displayed. The user is looking at an image. Don't take my word for it - open one of these and zoom to 1000%. It turns into a bunch of bricks. That's not bad, but it's not vector art.

For reference, the other mode is: Everything is digital vector-art. Suppose you make a reproduction of your document, starting with a blank Word document. You carefully match fonts and layout. And you hit "Print to PDF". Because the source data is digital, the fonts are stored as fonts, text as text etc. Text is never an image. No special snowflakes, every T is identical because it's rendered from the same font data. And if you zoom 1000%, it re-renders and the fonts are flawless. That would be wrong for a historic document, but, it's not our problem since we are not making documents that way.

Does that make things more clear?

Ask any archivist. An artifact you can't find is, literally, lost from the collection. Find-ability is mandatory. OCR is easy and it doesn't hurt your documents at all.