# WhatsApp CRM Logger — Full Project Specification
> Hand this file (plus the files in `/extension` and `/backend`) to Cursor to build the complete system.

---

## 1. What this system does

A Chrome extension silently captures every message visible in WhatsApp Web (web.whatsapp.com) — both incoming and outgoing — and ships the data to a backend PHP API. Nothing is stored in the browser beyond a short display log in the popup. The backend does all the heavy lifting: it persists every message, exposes a Logger UI for browsing those messages, and fires a webhook to a second module (Actions.php) that maps contacts and links messages to CRM records.

---

## 2. System components

```
┌─────────────────────────────────────────────────────────┐
│                   Chrome Extension                      │
│  content.js  →  background.js  →  POST whatsapplogger.php │
└─────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────┐
│              Backend: whatsapplogger.php                │
│  • Receives message from extension                      │
│  • Stores in whatsapplog table                          │
│  • Fires webhook → actions.php                          │
│  • Exposes REST-style endpoints for Logger UI           │
└─────────────────────────────────────────────────────────┘
                              │
              ┌───────────────┴──────────────┐
              ▼                              ▼
┌─────────────────────┐        ┌─────────────────────────┐
│   Logger UI         │        │      actions.php         │
│  (browser app)      │        │  Maps number → contact   │
│  Read-only viewer   │        │  Saves to CRM tables     │
└─────────────────────┘        └─────────────────────────┘
```

---

## 3. Chrome Extension

### 3.1 Files
- `manifest.json` — Manifest V3, targets `https://web.whatsapp.com/*`
- `content.js` — DOM observer that runs inside WhatsApp Web
- `background.js` — Service worker, handles all outbound HTTP
- `popup.html` / `popup.js` — On/off toggle, API URL config, last-30-messages log

### 3.2 What content.js must extract per message

| Field | Description |
|---|---|
| `message_id` | WhatsApp's internal `data-id` attribute on the message node — unique per message |
| `chat_name` | Display name shown in the chat header (contact name or group name) |
| `phone_number` | The phone number or WhatsApp ID of the contact. For 1:1 chats: extract from the `data-id` (format is typically `TIMESTAMP_PHONENUMBER@c.us` or `PHONENUMBER@c.us`). For groups: the group JID (e.g. `123456789@g.us`) |
| `sender` | Who sent the message. `"Me"` for outgoing; the contact/author name for incoming |
| `sender_phone` | Phone number of the sender specifically. For outgoing = the logged-in user's number (if detectable, else `"me"`). For incoming in groups = the individual member's JID from the message `data-id` |
| `direction` | `"incoming"` or `"outgoing"` |
| `message_type` | `"text"` (default). Extendable later to `"image"`, `"audio"`, `"document"` |
| `text` | Full text content of the message |
| `wa_timestamp` | Timestamp parsed from the message node if available, else `new Date().toISOString()` |
| `source` | Always `"whatsapp_web"` |
| `chat_type` | `"individual"` or `"group"` based on whether the JID ends in `@g.us` |

### 3.3 Phone number extraction
WhatsApp Web embeds the JID in the `data-id` attribute of each message node.
Format: `[epoch]_[JID]_[hash]` where JID = `5491112345678@c.us` (individual) or `1234567890-1234567890@g.us` (group).
The extension must parse this to extract the raw phone number / group ID and send it as `phone_number`.

### 3.4 Deduplication
Track `message_id` values in a `Set` in memory. Skip any message whose ID is already in the Set. This prevents re-sending when the user scrolls or when WhatsApp re-renders the DOM.
the `message_id` (the WhatsApp `data-id` attribute) is a real, stable, globally unique ID that WhatsApp generates. It must be use to implement the back-end detectino fo already store in table, by checking that this id was not sent and store already, and avoid duplciating it in the database

```
true_5491112345678@c.us_3EB0C4A1F2D8E6B7A9C1
                                └─ this part is a unique hash WhatsApp generates
```

So the deduplication strategy is already two-layered in the current design:

- **Layer 1 — extension (in-memory):** the `Set` in `content.js` prevents duplicate POSTs within a single session. Fast, no network cost. Resets on tab close — that's fine.
- **Layer 2 — database (permanent):** `whatsapplog` has `UNIQUE KEY uq_message_id (message_id)` and the insert uses `INSERT IGNORE`. So even if the extension sends the same message twice across sessions, the database will silently reject the duplicate. Nothing breaks, nothing gets logged twice.

This means the database is the real source of truth for deduplication, and the in-memory Set is just an optimization to avoid unnecessary network calls during a session. You're fully protected — no message will ever appear twice in `whatsapplog`.

### 3.5 Extension popup
The popup has three sections:
1. **Control** — on/off toggle. When off, content.js stops sending (but still observes so it doesn't miss messages when re-enabled).
2. **Config** — text input to save the `whatsapplogger.php` base URL. Saved to `chrome.storage.sync`.
3. **Live log** — last 30 messages received from `chrome.storage.local`, shown as a minimal list with chat name, direction arrow, truncated text, and sent/failed status badge.

### 3.6 background.js responsibilities
- Receives `LOG_MESSAGE` from content.js
- POSTs to `whatsapplogger.php` (endpoint: `POST /whatsapplogger.php`)
- On HTTP failure: retries up to 3 times with exponential backoff using `chrome.alarms`
- Appends result (sent/failed) to `chrome.storage.local` messageLog (capped at 100 entries)
- Optionally sends an `X-API-Key` header if one is configured in the popup

---

## 4. Backend: whatsapplogger.php

### 4.1 Purpose
Single PHP file (or a class/module) that acts as the central hub. It receives messages from the extension, stores them, exposes read endpoints for the Logger UI, and fires a webhook to `actions.php`.

### 4.2 Database table: `whatsapplog`

| Column | Type | Notes |
|---|---|---|
| `id` | INT UNSIGNED AUTO_INCREMENT PK | |
| `message_id` | VARCHAR(200) UNIQUE | WhatsApp's data-id — dedup key |
| `phone_number` | VARCHAR(50) | The contact/chat JID (e.g. `5491112345678` or group ID) |
| `chat_name` | VARCHAR(255) | Display name of the contact or group |
| `chat_type` | ENUM('individual','group') | |
| `sender` | VARCHAR(255) | Display name of who sent it |
| `sender_phone` | VARCHAR(50) | JID of the sender (useful in groups) |
| `direction` | ENUM('incoming','outgoing') | |
| `message_type` | VARCHAR(30) | `text`, `image`, `audio`, `document` |
| `message_text` | TEXT | Body of the message |
| `wa_timestamp` | DATETIME | Timestamp from WhatsApp |
| `received_at` | DATETIME DEFAULT NOW() | When the server received it |
| `webhook_sent` | TINYINT(1) DEFAULT 0 | Whether actions.php was notified |
| `webhook_sent_at` | DATETIME NULL | When the webhook completed |
| `webhook_response` | TEXT NULL | HTTP response body from actions.php (for debugging) |

### 4.3 Endpoints

All endpoints live in `whatsapplogger.php`, routed by the `action` query parameter (or a `switch` on `$_GET['action']`).

---

#### `POST /whatsapplogger.php`  ← called by the extension
**Action:** `action=log` (or default when no action given)

Accepts JSON body. Validates required fields. Inserts into `whatsapplog` using `INSERT IGNORE` (dedup via UNIQUE key). After a successful insert, immediately calls the internal `fireWebhook()` function to notify `actions.php`.

**Response:** `{ "ok": true, "id": 123 }` on success. `{ "ok": false, "error": "..." }` on failure.

---

#### `GET /whatsapplogger.php?action=list_messages`
Returns a paginated list of all messages, newest first.

Query params:
- `page` (int, default 1)
- `per_page` (int, default 50, max 200)
- `search` (string, optional) — filters on `message_text` or `chat_name` or `phone_number`
- `direction` (string, optional) — `incoming` or `outgoing`
- `phone_number` (string, optional) — filter to one contact

Response:
```json
{
  "ok": true,
  "total": 1452,
  "page": 1,
  "per_page": 50,
  "data": [ { ...message row... }, ... ]
}
```

---

#### `GET /whatsapplogger.php?action=list_contacts`
Returns all distinct `phone_number` values that appear in the log, with summary stats.

Response:
```json
{
  "ok": true,
  "data": [
    {
      "phone_number": "5491112345678",
      "chat_name": "John Doe",
      "chat_type": "individual",
      "total_messages": 42,
      "incoming": 30,
      "outgoing": 12,
      "last_message_at": "2025-01-15T14:32:00Z",
      "last_message_preview": "Can you send the quote?"
    },
    ...
  ]
}
```

---

#### `GET /whatsapplogger.php?action=get_conversation&phone_number=5491112345678`
Returns the full message thread for a given phone number (or group ID), ordered oldest-first (so the UI can render it top-to-bottom like a real chat).

Query params:
- `phone_number` (required)
- `page` (int, default 1)
- `per_page` (int, default 100)

Response:
```json
{
  "ok": true,
  "phone_number": "5491112345678",
  "chat_name": "John Doe",
  "total": 42,
  "data": [ { ...message row... }, ... ]
}
```

---

#### `GET /whatsapplogger.php?action=stats`
Returns aggregate numbers for the Logger UI dashboard.

Response:
```json
{
  "ok": true,
  "total_messages": 3201,
  "total_contacts": 87,
  "today": 45,
  "incoming": 1800,
  "outgoing": 1401
}
```

---

### 4.4 Webhook: fireWebhook() → actions.php

After every successful `INSERT` into `whatsapplog`, `whatsapplogger.php` fires an HTTP POST to `actions.php`. This call should be:
- **Asynchronous** where possible (use cURL with a very short timeout so `whatsapplogger.php` doesn't block waiting for `actions.php` to respond)
- **Retried** if `webhook_sent` is still 0 (a background cron or a retry on next insert can sweep failed ones)

**Payload sent to actions.php:**
```json
{
  "event":          "whatsapp_message",
  "log_id":         123,
  "message_id":     "3EB0C4A1F2...",
  "phone_number":   "5491112345678",
  "chat_name":      "John Doe",
  "chat_type":      "individual",
  "sender":         "John Doe",
  "sender_phone":   "5491112345678",
  "direction":      "incoming",
  "message_type":   "text",
  "message_text":   "Can you send me the quote?",
  "wa_timestamp":   "2025-01-15T14:32:00Z",
  "received_at":    "2025-01-15T14:32:01Z",
  "source":         "whatsapp_web"
}
```

After the cURL call, update `webhook_sent = 1`, `webhook_sent_at = NOW()`, and store the HTTP response body in `webhook_response`.

---

## 5. actions.php (existing module — interface spec only)

`whatsapplogger.php` calls `actions.php` via HTTP POST. `actions.php` is a separate, independently developed module. Its responsibilities are:

1. Receive the webhook payload from `whatsapplogger.php`
2. Look up `phone_number` in its own contacts table to find a matching CRM contact
3. If a match is found, insert a record into its own message/interaction log table, linked to that contact
4. If no match is found, optionally create a new lead/contact record, or queue it for manual review
5. Return `{ "ok": true }` or `{ "ok": false, "error": "..." }` to `whatsapplogger.php`

`actions.php` must accept `Content-Type: application/json` POST requests. It does **not** call `whatsapplogger.php` back.

---

## 6. Logger UI

The Logger UI is a standalone browser-based interface (separate HTML file or SPA) that reads from `whatsapplogger.php` endpoints. It is **read-only** — no sending of messages.

### 6.1 Views

**Dashboard / Home**
- Stats bar: total messages, total contacts, messages today, incoming vs outgoing counts
- Recent messages list (last 50), each row showing: chat name, direction badge, message preview, timestamp

**Contacts list**
- All phone numbers / contacts in the log
- Each row: avatar initial, chat name, phone number, total messages, last message preview, last activity timestamp
- Search box to filter by name or number
- Click a row → opens the Conversation view

**Conversation view**
- Renders messages for the selected phone number in a WhatsApp-style bubble layout:
  - Outgoing messages: right-aligned, green bubble
  - Incoming messages: left-aligned, white/light bubble
  - Date separators between days
  - Sender name label (for group chats)
  - Timestamp under each bubble
- Infinite scroll / load-more for pagination
- Header shows contact name, phone number, chat type badge

**Search**
- Full-text search across all messages
- Results show which conversation each message belongs to
- Click result → opens that conversation scrolled to that message

### 6.2 Technical notes for Cursor
- Can be implemented as plain HTML + vanilla JS (no framework required) or as a simple React/Vue SPA
- Calls the `whatsapplogger.php` endpoints directly via `fetch()`
- Should include an API base URL config (either hardcoded or a small config file)
- No authentication is specified here — the developer implementing it should add HTTP Basic Auth or token auth at the server level

---

## 7. Data flow summary

```
User opens a chat on web.whatsapp.com
        │
        ▼
content.js detects new message node in DOM
        │  Extracts: message_id, phone_number, chat_name, sender,
        │            direction, text, wa_timestamp, chat_type
        ▼
background.js receives LOG_MESSAGE event
        │  POST → whatsapplogger.php  (JSON body)
        │  Retry on failure (up to 3×, exponential backoff)
        │  Writes result to chrome.storage.local for popup display
        ▼
whatsapplogger.php receives the POST
        │  Validates fields
        │  INSERT IGNORE into whatsapplog
        │  Fires cURL POST → actions.php  (async, short timeout)
        │  Updates webhook_sent + webhook_response
        │  Returns { ok: true, id: ... }
        ▼
actions.php receives webhook
        │  Looks up phone_number in CRM contacts
        │  Links message to contact record
        │  Returns { ok: true }
        ▼
Logger UI (separate, on demand)
        │  GET whatsapplogger.php?action=list_contacts
        │  GET whatsapplogger.php?action=get_conversation&phone_number=...
        └─ Renders WhatsApp-style conversation view
```

---

## 8. Configuration values (to be set during setup)

| Setting | Where set | Description |
|---|---|---|
| `whatsapplogger.php` base URL | Extension popup | URL the extension POSTs to |
| `ACTIONS_PHP_URL` | whatsapplogger.php config block | Full URL to actions.php webhook |
| `API_KEY` | whatsapplogger.php + extension popup | Shared secret sent as `X-API-Key` header |
| DB credentials | whatsapplogger.php config block | MySQL host, db, user, password |
| `WEBHOOK_TIMEOUT_SECONDS` | whatsapplogger.php config block | cURL timeout for actions.php call (recommended: 2s) |

---

## 9. File structure for Cursor

```
/
├── SPEC.md                         ← this file
│
├── extension/
│   ├── manifest.json
│   ├── content.js
│   ├── background.js
│   ├── popup.html
│   ├── popup.js
│   └── icons/
│       ├── icon16.png
│       ├── icon48.png
│       └── icon128.png
│
└── backend/
    ├── whatsapplogger.php          ← main API module
    ├── schema.sql                  ← DB table definition
    ├── actions.php                 ← stub / interface (existing module)
    └── logger-ui/
        ├── index.html              ← Logger UI entry point
        ├── app.js                  ← UI logic
        └── style.css               ← WhatsApp-style chat UI
```