Converting a PDF to an EPUB using InDesign

Question

I have four finished books in PDF files, totaling 1857 pages including over 600 photographs, which I need to convert to EPUB files in InDesign.  I used a script to import all of the pages of one book into InDesign into a document reflecting the page and margin parameters of the original Microsoft Word document file.  The imported PDF looked perfect in InDesign once I switched to a high-quality display performance in the View menu.  However, when I exported the document into an EPUB file, I ran into two problems that I cannot fix. 
Problem #1:  Both the Digital Editions EPUB reader and the Nook recognize the EPUB document as having only two pages (displaying 1 / 2 on the screen, even though I can scroll to or page to all 430 pages in both Digital Editions and the Nook.  This happens despite the fact that the correct number of pages correctly displays in the InDesign document settings.
Problem #2:  The document text in the EPUB file is so poorly rendered it is almost illegible.  Each letter is chunky and fragmented and partially pixelated.  However, the same text in InDesign perfectly reflects the high quality fonts used in the original PDF and Microsoft Word Documents.
I can find no documentation in the InDesign help system to explain why I am getting these results.  When I cut and paste a few pages of text from the PDF into an InDesign layout, it looks fantastic when I export it to an EPUB. But the text in the script loaded PDF pages look horrible.  And for what it is worth, Problems #1 and #2 persisted, even when bypassed the script and I hand placed a half dozen pages from the PDF into an InDesign document and exported them to an EPUB file. 
Also, if I export back to a PDF from InDesign, the text looks perfect and the page numbering is correct in Digital Editions.  The only reference I found related to InDesign text quality on export  is in Object Export Options, but they remain disabled to me.
And as a final test, I script loaded all 710 pages of the indesign_cs5_help.pdf to see if it would display the same problems. Not surprisingly, the text was just as poorly rendered in Digital Editions, which then displayed the first page of the document as 1 / 3 instead of 1 / 710.
Is there some method I can use to fix these problems?  Or am I going to have to recreate the layout for all 1857 pages painstakingly object by object in InDesign?
Thanks for all assistance.

Answer 1

I have managed to work out the entire process over the past three weeks both working with InDesign with EPUB and by helping a friend post a series of books directly from Microsoft Word via MOBI to Amazon.  I will discuss the latter process first to point out the inherent problems with the direct approach, so the reasons for using InDesign become more apparent.
From Microsoft Word
First, I recommend you reformat your Word document into a smaller book size to simulate the maximum text display you will get on a Kindle (3.0 in. x 5.3 in. margins) or Nook (3.3 in. X 5.5 in. margins).  Next, make sure you are using style tags and paragraph and page layout options everywhere (no tabs or spaces in formatting), and a TOC object for the table of contents.  All photos or images must be centered, and if you want them to have horizontal width, then stretch them from side to side.  Leave your cover photo out of the document, and add it via the author panel specifying you want the uploaded cover image to be used inside the book.  This will assure that the cover photo will correctly fill the frame.
Next save your DOCX document as a FILTERED HTML.  Then close Word and load the HTML into the Kindle Previewer to review to verify your images are correctly displayed.  When you do this, it creates a MOBI file that you can upload to a Kindle to view first hand.  Then when you are satisfied with the results, you can upload this MOBI book file to Amazon Authors Direct.  The upload dialog does not say it will accept a MOBI file, but the HTML file it recommends does not contain any image data.  As it accepts an EPUB file, it only made sense it would accept the MOBI file, and it does (even the Amazon technical staff did not know this).
Using InDesign
The results of going from Word direct to MOBI are far from perfect.  But I would judge they are minimally acceptable.  However, if you want complete control of a book’s layout I recommend going the InDesign route.  Without exception, the pages, text and images laid out in InDesign are (or will be soon) perfectly reflected on the Nook and Kindle.  Granted, it takes time to lay out a book page by page, but if you do it right by using the InDesign paragraph and object styles, the process goes quickly. 
Create an InDesign Book with each chapter a separate document in the book to ensure chapters start on a new page.
Make document pages small to maintain the look and feel.  They need not be 3.5 x 5.5.  I used a 5 x 7 inch document page and everything looked fine on the Kindle and Nook.  Get used to the idea that the text will always flow differently based on the text size selected, So only worry about inline image placement, chapter headings and such, and leave the rest to fate.
After you get one chapter finished, use it to create a template document to use for subsequent chapter documents to facilitate laying out your book.  A photo in a graphic frame with an anchored caption text box is easily copied and pasted elsewhere to standardize all image objects.  Be sure to synchronize styles and swatches for each new chapter added.
InDesign footnoting did not work correctly in the EPUB for me, but I found it easy to use the hyperlink system to emulate footnoting functionality.  By placing the footnotes at the back of the book and implementing a hyperlink back to the inclusion point of the footnote in a chapter, the reader can use the footnote pages like a hot index as well as a hot footnote.
When generating an EPUB,
Select No cover image
Select ordering same as Article panel
Set Book margin to 0
Image size relative to page
Contents XHTML
Use InDesign TOC (set style used)
Generate CSS with all boxes checked.
One thing to be aware of is that the digital readers will read the HTML page objects left to right then top to bottom, so I recommend the following:
In InDesign Guides and Pasteboard preferences, set the Snap to Zone to 1 to guarantee all page objects line up on the left margin exactly.  Manually adjust each object box X location as necessary.  Lock object when completed to minimize inadvertent movement
Always guarantee all text box objects stretch from left to right margin.  Use the Indent controls to vary text placement and make this a new paragraph style.
Always anchor a photo (inset in a graphic frame) to a caption text box stretched margin-to-margin to ensure the photos will not sink to the bottom of a page in the EPUB. 
Photos snapped to each margin will display full width and will display larger when the Kindle is rotated 90 degrees.
Add 0.25 in, Space After on the last paragraph before an image and 0.25 in, Space Before on the first paragraph after an image to create a bit of white space around the image and caption.
Always use the Article list to verify the placement of page objects in each chapter document.  I have had a few images sink when the article list is generated despite all the above precautions.  But it is easy to manually make adjustments.
Kindle Tools
Amazon released a series of tools and PDF documents a few weeks ago that greatly simplify the process of posting a book for sale via Direct Publishing.  That said, rabbit holes abound waiting to catch anyone who is not already familiar with HTML and document layout software technologies.  So the non-tech masses should be wary.
The Amazon Tools are at https://kdp.amazon.com/self-publishing/help?topicId=A3IWA2TQYMZ5J6 .
KindleGen v2.4 – a command line tool used to build eBooks that can be sold through Amazon's Kindle platform. This tool is best for publishers and individuals who are familiar with HTML and want to convert their HTML, XHTML, XML (OPF/IDPF format), or ePub source into a Kindle Book.
Kindle Previewer v2.4 -- a tool that emulates how books display on Kindle devices and applications. Kindle Previewer makes it easy to preview the layout of a book and make sure its text displays properly for any orientation or font size. This tool is recommended for publishers, eBook conversion companies, and individual authors in combination with KindleGen to produce the highest quality Kindle books. It is available for both Windows and Mac OS X platforms.
Kindle Plugin for Adobe InDesign® v0.96 (Beta) --  a plug-in which can be used to convert documents or books created in Adobe InDesign® to Kindle format.  For more information on creating books for the Kindle platform from Adobe InDesign®, please see the Publishing Guidelines for Kindle Plugin for Adobe InDesign® available for download in PDF, Kindle and Adobe InDesign® formats.
I found that the Previewer and the DOS KindleGen produce identical MOBI files, so I recommend the previewer for its ease of use.  After extensive testing I have found the Kindle InDesign plugin will perfectly convert the text from InDesign; however, it fails to size, place and display photographs and captions in all cases.  It also does not render the InDesign TOC correctly.  But remember, it is Beta software .  I sent in a bug list to Amazon via the Export Book to Kindle dialog in InDesign and never got a response, but presumably someone is working on the problems and will eventually get them fixed.  On the other hand, currently the Previewer perfectly displays photographs in a MOBI file when the InDesign EPUB file is fed into it; and the only problem with the text is that the line spacing is incorrect so paragraph lines are somewhat scrunched together.  While this is not a problem for developing a book, I would wait until the final release of the Kindle Plugin for InDesign before generating and publishing a MOBI book.
I hope this helps.