# XML Parsing: Handle XML Data

# XML Parsing: Handle XML Data & XML to JSON Conversion

## Problem

XML is a widely-used data format, but JSON is often more convenient for modern applications. We need to:
1. Parse XML data correctly
2. Handle nested structures, attributes, and text content
3. Convert XML to JSON format
4. Handle edge cases (empty elements, special characters, namespaces)

## Solution

### Approach

1. **Parse XML** using a robust XML parser
2. **Traverse the tree** recursively to build JSON structure
3. **Handle attributes** by prefixing with `@` or nesting in special keys
4. **Manage text content** and mixed content scenarios
5. **Normalize** the output for consistency

### Key Considerations

- **Attributes vs Elements**: Decide how to represent XML attributes in JSON
- **Text Nodes**: Handle both element text and mixed content
- **Arrays**: Detect repeated elements and convert to arrays
- **Namespaces**: Strip or preserve namespace prefixes
- **Empty Elements**: Represent as `null`, empty string, or empty object

---

## Code

### 1. Basic XML to JSON Converter (Python)

```python
import xml.etree.ElementTree as ET
import json
from typing import Any, Dict, List, Union

class XMLToJSONConverter:
    """Convert XML to JSON with flexible options"""
    
    def __init__(self, include_attributes: bool = True, 
                 array_detection: bool = True,
                 text_key: str = "#text",
                 attr_prefix: str = "@"):
        self.include_attributes = include_attributes
        self.array_detection = array_detection
        self.text_key = text_key
        self.attr_prefix = attr_prefix
    
    def parse(self, xml_string: str) -> Dict[str, Any]:
        """Parse XML string and return JSON-compatible dict"""
        try:
            root = ET.fromstring(xml_string)
            return {root.tag: self._element_to_dict(root)}
        except ET.ParseError as e:
            raise ValueError(f"Invalid XML: {e}")
    
    def _element_to_dict(self, element: ET.Element) -> Any:
        """Recursively convert XML element to dict"""
        result = {}
        
        # Add attributes
        if self.include_attributes and element.attrib:
            for key, value in element.attrib.items():
                result[f"{self.attr_prefix}{key}"] = value
        
        # Process children
        children = {}
        for child in element:
            child_data = self._element_to_dict(child)
            
            if child.tag in children:
                # Convert to list if multiple children with same tag
                if not isinstance(children[child.tag], list):
                    children[child.tag] = [children[child.tag]]
                children[child.tag].append(child_data)
            else:
                children[child.tag] = child_data
        
        # Add text content
        text = element.text.strip() if element.text else ""
        tail = element.tail.strip() if element.tail else ""
        
        # Combine results
        if children:
            result.update(children)
            if text:
                result[self.text_key] = text
        elif text:
            # Only text, no children
            if result:  # Has attributes
                result[self.text_key] = text
            else:
                return text
        elif result:
            # Has attributes but no text/children
            pass
        else:
            # Empty element
            return None
        
        return result if result else None
    
    def to_json(self, xml_string: str, indent: int = 2) -> str:
        """Convert XML to JSON string"""
        data = self.parse(xml_string)
        return json.dumps(data, indent=indent, ensure_ascii=False)


# Example Usage
if __name__ == "__main__":
    xml_data = """<?xml version="1.0"?>
    <library>
        <book id="1" author="John Doe">
            <title>Python Guide</title>
            <year>2023</year>
            <price currency="USD">29.99</price>
        </book>
        <book id="2" author="Jane Smith">
            <title>Web Development</title>
            <year>2024</year>
            <price currency="USD">39.99</price>
        </book>
        <book id="3">
            <title>Data Science</title>
        </book>
    </library>
    """
    
    converter = XMLToJSONConverter()
    json_output = converter.to_json(xml_data)
    print(json_output)
```

**Output:**
```json
{
  "library": {
    "book": [
      {
        "@id": "1",
        "@author": "John Doe",
        "title": "Python Guide",
        "year": "2023",
        "price": {
          "@currency": "USD",
          "#text": "29.99"
        }
      },
      {
        "@id": "2",
        "@author": "Jane Smith",
        "title": "Web Development",
        "year": "2024",
        "price": {
          "@currency": "USD",
          "#text": "39.99"
        }
      },
      {
        "@id": "3",
        "title": "Data Science"
      }
    ]
  }
}
```

---

### 2. Advanced XML Parser with Namespace Handling

```python
import xml.etree.ElementTree as ET
from collections import defaultdict
from typing import Any, Dict, Optional

class AdvancedXMLParser:
    """Handle namespaces, CDATA, and complex structures"""
    
    def __init__(self, strip_namespaces: bool = True):
        self.strip_namespaces = strip_namespaces
    
    def _strip_namespace(self, tag: str) -> str:
        """Remove namespace prefix from tag"""
        if self.strip_namespaces and '}' in tag:
            return tag.split('}', 1)[1]
        return tag
    
    def parse_file(self, filepath: str) -> Dict[str, Any]:
        """Parse XML from file"""
        tree = ET.parse(filepath)
        root = tree.getroot()
        return {self._strip_namespace(root.tag): self._parse_element(root)}
    
    def parse_string(self, xml_string: str) -> Dict[str, Any]:
        """Parse XML from string"""
        root = ET.fromstring(xml_string)
        return {self._strip_namespace(root.tag): self._parse_element(root)}
    
    def _parse_element(self, element: ET.Element) -> Any:
        """Parse single element"""
        tag = self._strip_namespace(element.tag)
        
        # Collect attributes
        attrs = {f"@{k}": v for k, v in element.attrib.items()}
        
        # Collect children
        children = defaultdict(list)
        for child in element:
            child_tag = self._strip_namespace(child.tag)
            child_data = self._parse_element(child)
            children[child_tag].append(child_data)
        
        # Get text content
        text = (element.text or "").strip()
        
        # Build result
        result = {}
        
        # Add attributes
        result.update(attrs)
        
        # Add children (convert single-item lists to single values)
        for child_tag, child_list in children.items():
            result[child_tag] = child_list[0] if len(child_list) == 1 else child_list
        
        # Add text
        if text:
            if result:
                result["#text"] = text
            else:
                return text
        
        return result if result else None


# Example with namespaces
xml_with_ns = """<?xml version="1.0"?>
<root xmlns:app="http://example.com/app">
    <app:user id="1">
        <app:name>John</app:name>
        <app:email>john@example.com</app:email>
    </app:user>
</root>
"""

parser = AdvancedXMLParser(strip_namespaces=True)
result = parser.parse_string(xml_with_ns)
print(json.dumps(result, indent=2))
```

---

### 3. Using Popular Library: `xmltodict`

```python
import xmltodict
import json

# Simple one-liner conversion
xml_string = """<?xml version="1.0"?>
<root>
    <person>
        <name>Alice</name>
        <age>30</age>
    </person>
</root>
"""

# Parse XML to OrderedDict
data = xmltodict.parse(xml_string)

# Convert to JSON
json_output = json.dumps(data, indent=2)
print(json_output)

# With options
data = xmltodict.parse(
    xml_string,
    attr_prefix='@',           # Attribute prefix
    cdata_key='#text',         # CDATA key
    force_list=('person',),    # Force these as lists
    strip_whitespace=True      # Strip whitespace
)
```

---

### 4. Bidirectional Conversion (XML ↔ JSON)

```python
import xml.etree.ElementTree as ET
import json

class XMLJSONConverter:
    """Convert between XML and JSON bidirectionally"""
    
    @staticmethod
    def json_to_xml(json_data: Dict, root_name: str = "root") -> str:
        """Convert JSON dict to XML string"""
        root = ET.Element(root_name)
        XMLJSONConverter._dict_to_element(root, json_data)
        return ET.tostring(root, encoding='unicode')
    
    @staticmethod
    def _dict_to_element(parent: ET.Element, data: Any) -> None:
        """Recursively convert dict to XML elements"""
        if isinstance(data, dict):
            for key, value in data.items():
                if key.startswith('@'):
                    # Attribute
                    parent.set(key[1:], str(value))
                elif key == '#text':
                    # Text content
                    parent.text = str(value)
                else:
                    # Child element
                    if isinstance(value, list):
                        for item in value:
                            child = ET.SubElement(parent, key)
                            XMLJSONConverter._dict_to_element(child, item)
                    else:
                        child = ET.SubElement(parent, key)
                        XMLJSONConverter._dict_to_element(child, value)
        else:
            parent.text = str(data)
    
    @staticmethod
    def xml_to_json(xml_string: str) -> str:
        """Convert XML to JSON"""
        root = ET.fromstring(xml_string)
        data = {root.tag: XMLJSONConverter._element_to_dict(root)}
        return json.dumps(data, indent=2)
    
    @staticmethod
    def _element_to_dict(element: ET.Element) -> Any:
        """Convert element to dict"""
        result = {}
        
        # Attributes
        for key, value in element.attrib.items():
            result[f"@{key}"] = value
        
        # Children
        for child in element:
            if child.tag in result:
                if not isinstance(result[child.tag], list):
                    result[child.tag] = [result[child.tag]]
                result[child.tag].append(XMLJSONConverter._element_to_dict(child))
            else:
                result[child.tag] = XMLJSONConverter._element_to_dict(child)
        
        # Text
        text = (element.text or "").strip()
        if text:
            if result:
                result['#text'] = text
            else:
                return text
        
        return result if result else None


# Test bidirectional conversion
json_data = {
    "person": {
        "@id": "1",
        "name": "Bob",
        "age": "25"
    }
}

# JSON to XML
xml_string = XMLJSONConverter.json_to_xml(json_data, "root")
print("XML:", xml_string)

# XML to JSON
json_output = XMLJSONConverter.xml_to_json(xml_string)
print("JSON:", json_output)
```

---

## Summary Table

| Approach | Pros | Cons | Use Case |
|----------|------|------|----------|
| **ElementTree** | Built-in, lightweight | Limited features | Simple XML parsing |
| **Custom Parser** | Full control, flexible | More code | Complex requirements |
| **xmltodict** | Simple, intuitive | Less control | Quick conversions |
| **lxml** | Fast, powerful | External dependency | Large XML files |
| **Bidirectional** | Reversible | Complex logic | Data synchronization |

---

## Best Practices

✅ **Do:**
- Validate XML before parsing
- Handle namespaces explicitly
- Use appropriate data types
- Test edge cases (empty elements, special chars)
- Document attribute representation strategy

❌ **Don't:**
- Assume all XML is well-formed
- Ignore whitespace handling
- Lose information during conversion
- Use regex for XML parsing
- Forget to escape special characters