How to get a GREP. Not a typo. This is Real and Powerful

Now that myTweetPack.com is pretty much done, (Or as done as any major software ever is) I'm turning my attention back to self-publishing. 

Specifically, I'm looking at Amazon publishing. 

After all, Amazon is responsible for 70% of all books sold. Not just online books . . . ALL books. That's a market that begs investigation.

While digging into the technical side of creating an eBook, I saw a need to fiddle with the HTML Microsoft Word poops out. 

Was that too harsh? I don't think so.

Issue #1: Amazon prefers to start from HTML exported from Microsoft Word.

OMG

Finagling a 17,000 to 70,000-word eBook pooped out as HTML by Word is a minefield of Search and Replace functions that can get you into serious trouble in an awful hurry.

That's when you need to get a GREP

This post is a primer on Regular Expressions which are not so "regular" at all.

GREP is a nickname for Regular Expressions. The term is usually shortened to "regex" or "regexp." Some people, cruel ones, even call them "rational expressions."

GREP has a lot more uses than just cleaning up poopy HTML. Regex is common in data validation for web forms.

GERP is a search "language," or a pattern-matching scheme. It gives superpowers to things like Find and Replace functions.

To activate GREP in Word, check "Use wildcards" in the dialog box. Turn it on when you need it, and back off when you don't. Warning: Word only allows a subset of GREP.

Better yet, download Notepad++, a free code editor and use GREP at full-throttle.

What, the Hell, is GREP?

Word gives us a hint when it offers to "Use wildcards." Regular Expressions are a search language, a set of wildcards if you will. They are simple strings (text) that are really a search function on more steroids than a Russian weight-lifter.

This ain't a simple subject. Believe me. 

This is as simple as it can get.

I can see my fabourite technophobes, Lisa Gallagher and Trent Selbrede,  both falling to the floor in complete catatonic states as they read this.

My apologies to their families and other loved ones.

I'd miss them too.

Trent, Lisa, please stop reading now.

Seriously, GREP really is the "easier" way to fix Word's HTML-poop. The alternative would be to use a few hundred search-and-replace commands that would need be done in the correct order.

That would be a nightmare.

The "language" looks weird but isn't very tough to learn. It can make your search and replace actions lightning fast and super-easy. 

I cleaned up a 26,000-word manuscript in under 3 seconds.

Practice on something unimportant, or on a copy of the actual document.

WARNING: If you're new to Regular Expressions, and even if you're not, never work on the document itself. Work on a copy. Regex search and replace is lightning fast. It's very flexible, but it can't read minds. It will do what you tell it to do even if you made a mistake. It can fix a 300-page manuscript in seconds. It can destroy it just as fast.

Strictly speaking, regex is case-sensitive. "Paul" is not the same as "paul." That isn't true for most code editors. Ex: Notepad++ defaults to case insensitive. 

Regex is all about pattern matching

You already know about pattern matching. If you want to find all txt files in a folder, you just ask for *.txt. You know that means, "Show me files that start with anything and end in .txt." In Windows, * is a placeholder for "anything."

Regex kicks it up a notch or twenty. 

In case you're curious, the regex equivalent of *.txt is  ^.*\.txt$

You tell Regex what pattern you want to find. It goes out and finds them.

The rules are simple. 

So are the rules for chess. 

Simplicity often masks complexity.

You can make a single super-complicated regex or a few simple ones. Your choice.

Even ten or so regex strings is better than a couple of hundred search-replace commands.

REGEX in a nutshell

• Any letter, digit or symbol will be matched as is. It's important to remember that some symbols have special meaning. They can't be used as-is. Technically, they are called "metacharacters." Think of them as "reserved." The metacharacters are the backslash \, the caret ^, the dollar sign $, the period or dot ., the vertical bar or pipe symbol |, the question mark ?, the asterisk or star *, the plus sign +, the opening parenthesis (, the closing parenthesis ), the opening square bracket [, and the opening curly brace {. Type a \ before them to let regex know that you want to treat a metacharacter as a literal value. Don't use 1+2=3. Use 1\+2=3.
• Some letters are shortcuts. You turn them into shortcuts by adding a \. Ex: \t is a tab.
• Square brackets [ ] are a choice of characters. Gr[ae]y will match both "Gray" and "Grey." You can also use a ^ after the opening square bracket. A ^ after a square bracket means "not."
• A hyphen is a range. [0-9] will match any digit. [a-zA-Z] will match any character, upper or lower case.
• A period is a placeholder for anything. So, r.d will match "rid," "red," or even "r&d." 
• A caret (^) means "not" when inside square brackets. It turns inclusive searches into exclusive searches. ^P would match everything except P. P[^aul] would not match Paul. Outside of square brackets, it means "start with the first letter." So, ^a wouldn't match Paul because the "a" is not in the first position.
• Plus signs (+) mean one or more. [0-9]+ means any number of any digit in a row. Use {} for a specific number. [0-9]{3} means any 3 digits in a row.
• An asterix (*) means zero or more. [0-9]* means any digit or none.  <[A-Za-z][A-Za-z0-9]*> will match any HTML tag without attributes.
• A question mark (?) means zero or one. You can use that to match Britsh and US spellings. Ex: colou?r matches both "color" and "colour."
• The pipe character (|) means OR. You can also group them. To match "cat food" or "dog food," you'd use (cat|dog) food. You can add as many |'s as you want. That means you can also use (cat|dog|horse|rabbit|lemur/fish) food. Gerbils would be SOL.
• ANCHORS: Anchors don't match anything. They are for positions. For example, \b matches by word. See the next item for an example.
• Curly braces "{}" are for repetition. [0-9]{3} looks for any three digits in a row. It will match both "Testing123" and "123." To only match the digits, tell regex to only find them alone as a word. . . add \b. Like this, \b[0-9]{3}\b
• GROUPING, CAPTURING, AND BACK REFERENCES: Round brackets group a regex together. Once they are grouped, you can refer to them by number. (regex-a) (regex-b) can now be \1 for whatever regex-a is and \2 for regex-b. So, in Notepad++ I can use this in a Find and Replace: Find - ((a-zA-Z]+)[0-9](a-zA-Z]+) Replace - \1\2. That would look for one or more letters on either side of any digit and replace the whole shebang with just the text part. Any number of any letters on either side of a digit would be replaced. I don't need to know which letters or which numbers. I don't even need to know how many letters.

What do you think this regex matches? 

 \b[1-9][0-9]{3}\b   

Because of the "\b's"  it looks for a whole word. First, it looks for any digit from 1 through 9, then it looks for any three digits in a row from 0 through 9. 

Answer: This regex will match the numbers 1000 through 9999, but only if they are on their own. I.E. "1000" not "file_1000."

Here's some homework. What do these regex' match?

\b[1-9][0-9]{2,4}\b

or, how about this more complex but much more comon one?

 \b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b 

To dig deeper into Regex

For more info on Regex, including a full tutorial, visit Regular-Expressions.info