# Regex Tutorial: Master Regular Expressions for Text Processing

# Regex Tutorial: Master Regular Expressions for Text Processing

Pattern matching that saves hours of coding

## Why This Skill Matters

Regular expressions (regex) are the Swiss Army knife of text processing. Whether you're validating email addresses, extracting data from logs, or performing complex find-and-replace operations, regex skills separate efficient developers from those who waste hours on manual text manipulation.

**Real-world impact:**
- **Data validation**: Instantly verify email formats, phone numbers, and URLs
- **Log analysis**: Extract critical information from thousands of lines in seconds
- **Content migration**: Transform data between formats without manual intervention
- **Code refactoring**: Find and replace complex patterns across entire codebases
- **Web scraping**: Extract structured data from unstructured HTML

Learning regex typically takes 2-3 hours but saves 100+ hours annually for developers who process text regularly.

## Getting Started

### Understanding the Basics

Regex works by matching character patterns. Think of it as a search language with superpowers.

**Core concept**: A regex pattern is a sequence of characters that defines a search rule.

```
Pattern: cat
Text: "The cat sat on the mat"
Result: Matches "cat" (appears twice)
```

### Your First Patterns

**Literal matching** (simplest form):
```regex
hello
```
Matches the exact word "hello"

**Character classes** (match any character inside brackets):
```regex
[aeiou]
```
Matches any single vowel

**Negated character class**:
```regex
[^0-9]
```
Matches any character that's NOT a digit

**Ranges**:
```regex
[a-z]      # lowercase letters
[A-Z]      # uppercase letters
[0-9]      # digits
[a-zA-Z0-9]  # alphanumeric
```

### Metacharacters: Your Power Tools

| Metacharacter | Meaning | Example |
|---|---|---|
| `.` | Any character except newline | `a.c` matches "abc", "adc" |
| `*` | 0 or more of previous | `ab*c` matches "ac", "abc", "abbc" |
| `+` | 1 or more of previous | `ab+c` matches "abc", "abbc" (not "ac") |
| `?` | 0 or 1 of previous | `ab?c` matches "ac", "abc" |
| `^` | Start of line | `^hello` matches "hello" at line start |
| `$` | End of line | `world$` matches "world" at line end |
| `\` | Escape special chars | `\.` matches literal period |
| `\|` | OR operator | `cat\|dog` matches "cat" or "dog" |

## Essential Commands/Shortcuts

### Common Regex Patterns (Copy-Paste Ready)

**Email validation**:
```regex
^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$
```

**Phone number (US format)**:
```regex
^\(?([0-9]{3})\)?[-. ]?([0-9]{3})[-. ]?([0-9]{4})$
```

**URL extraction**:
```regex
https?://[^\s]+
```

**IPv4 address**:
```regex
\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b
```

**Hex color code**:
```regex
#(?:[0-9a-fA-F]{3}){1,2}
```

**Date (YYYY-MM-DD)**:
```regex
^\d{4}-(?:0[1-9]|1[0-2])-(?:0[1-9]|[12]\d|3[01])$
```

**Time (HH:MM:SS)**:
```regex
^([01]?[0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9]$
```

**HTML tags**:
```regex
<[^>]+>
```

**Whitespace cleanup** (multiple spaces to single):
```regex
\s+
Replace with: (single space)
```

### Quantifiers Cheat Sheet

```regex
{n}      # Exactly n times
{n,}     # n or more times
{n,m}    # Between n and m times
*        # {0,} - zero or more
+        # {1,} - one or more
?        # {0,1} - zero or one
```

**Example**: `a{2,4}` matches "aa", "aaa", or "aaaa"

### Anchors and Boundaries

```regex
^        # Start of string
$        # End of string
\b       # Word boundary
\B       # Non-word boundary
```

**Practical example**:
```regex
\bword\b
```
Matches "word" as complete word, not "sword" or "wording"

## Advanced Techniques

### Grouping and Capturing

Parentheses create groups for complex patterns:

```regex
(cat|dog)
```
Matches "cat" or "dog"

```regex
(\d{3})-(\d{3})-(\d{4})
```
Captures phone number parts separately

**Backreferences** (refer to captured groups):
```regex
(\w+)\s+\1
```
Matches repeated words like "hello hello"

### Lookahead and Lookbehind

**Positive lookahead** `(?=...)` - Assert what follows:
```regex
\d+(?=px)
```
Matches numbers followed by "px" (but doesn't include "px")

**Negative lookahead** `(?!...)` - Assert what doesn't follow:
```regex
\d+(?!px)
```
Matches numbers NOT followed by "px"

**Positive lookbehind** `(?<=...)` - Assert what precedes:
```regex
(?<=\$)\d+
```
Matches numbers preceded by "$"

**Negative lookbehind** `(?<!...)` - Assert what doesn't precede:
```regex
(?<!\$)\d+
```
Matches numbers NOT preceded by "$"

### Non-Capturing Groups

```regex
(?:cat|dog)
```
Groups without creating a capture (more efficient)

### Case-Insensitive Matching

```regex
(?i)pattern
```
Or use the `/i` flag depending on your language

### Multiline Mode

```regex
(?m)^pattern$
```
Makes `^` and `$` match line boundaries, not just string boundaries

## Practice Drills

### Drill 1: Extract Email Addresses
**Task**: Find all emails in text
```
Contact: john@example.com or support@company.co.uk
```
**Pattern**:
```regex
[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}
```

### Drill 2: Validate Strong Passwords
**Requirements**: 8+ chars, uppercase, lowercase, number, special char
```regex
^(?=.*[a-z])(?=.*[A-Z])(?=.*\d)(?=.*[@$!%*?&])[A-Za-z\d@$!%*?&]{8,}$
```

### Drill 3: Extract Markdown Links
**Task**: Find `[text](url)` patterns
```regex
\[([^\]]+)\]\(([^)]+)\)
```
Captures link text in group 1, URL in group 2

### Drill 4: Remove HTML Tags
**Pattern**:
```regex
<[^>]*>
```
Replace with nothing to strip all tags

### Drill 5: Format Phone Numbers
**Input**: `1234567890`
**Pattern**: `(\d{3})(\d{3})(\d{4})`
**Replace with**: `($1) $2-$3`
**Output**: `(123) 456-7890`

## Integration with Workflow

### In Popular Tools

**VS Code Find/Replace**:
1. Press `Ctrl+H` (Cmd+H on Mac)
2. Click the `.*` button to enable regex
3. Enter pattern and replacement

**JavaScript**:
```javascript
const email = "user@example.com";
const pattern = /^[^\s@]+@[^\s@]+\.[^\s@]+$/;
console.log(pattern.test(email)); // true

const text = "cat cat dog";
console.log(text.replace(/(\w+)\s+\1/g, "$1")); // "cat dog"
```

**Python**:
```python
import re

emails = re.findall(r'[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}', text)

result = re.sub(r'\s+', ' ', text)  # Normalize whitespace
```

**SQL**:
```sql
SELECT * FROM users WHERE email REGEXP '^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$';
```

**Command Line (grep)**:
```bash
grep -E '^[0-9]{3}-[0-9]{3}-[0-9]{4}$' phonelist.txt
```

## Pro Tips

**1. Start Simple, Build Complexity**
Don't write the entire pattern at once. Test components individually.

**2. Use Online Testers**
[regex101.com](https://regex101.com) and [regexr.com](https://regexr.com) provide real-time feedback and explanations.

**3. Escape Special Characters**
When matching literal special characters, prefix with backslash: `\.` for period, `\$` for dollar sign.

**4. Use Raw Strings**
In most languages, use raw strings to avoid double-escaping:
```javascript
const pattern = /\d+/;  // JavaScript
```
```python
pattern = r'\d+'  # Python
```

**5. Performance Matters**
- Use specific patterns instead of `.*` when possible
- Avoid excessive backtracking with greedy quantifiers
- Use non-capturing groups `(?:...)` when you don't need captures

**6. Test Edge Cases**
- Empty strings
- Very long strings
- Special Unicode characters
- Boundary conditions

**7. Document Complex Patterns**
Add comments explaining your regex:
```regex
(?x)
^                          # Start of string
[a-zA-Z0-9._%+-]+         # Local part
@                          # At symbol
[a-zA-Z0-9.-]+            # Domain
\.                         # Dot
[a-zA-Z]{2,}              # TLD
$                          # End of string
```

**8. Leverage Flags**
- `g` (global): Find all matches, not just first
- `i` (case-insensitive): Ignore case
- `m` (multiline): Treat each line separately
- `s` (dotall): Make `.` match newlines

## Summary

Regular expressions transform text processing from tedious manual work into elegant, automated solutions. Master these fundamentals:

✓ **Metacharacters**: `.`, `*`, `+`, `?`, `^`, `$`
✓ **Character classes**: `[abc]`, `[a-z]`, `[^0-9]`
✓ **Quantifiers**: `{n}`, `{n,m}`, `+`, `*`, `?`
✓ **Grouping**: `()` for capture, `(?:)` for non-capture
✓ **Anchors**: `^` and `$` for position matching
✓ **Advanced**: Lookahead/lookbehind for context-aware matching

**Next steps**:
1. Bookmark regex101.com for quick reference
2. Practice with real data from your projects
3. Build a personal library of patterns you use frequently
4. Explore language-specific regex features

The investment in regex mastery pays dividends throughout your career. What once took hours of manual work now takes minutes of pattern matching.