# WhatsApp Logger — System overview

This document explains how the WhatsApp Logger pieces work together: the Chrome extension, the CRM API, the Logger UI, and the Actions module.

For field-level API details and original product spec, see [`SPEC.md`](SPEC.md).

---

## 1. What the system does

When someone uses **WhatsApp Web** in Chrome with the extension installed:

1. New chat messages are detected in the browser.
2. Each message is sent to your **crmapi** server (`WhatsAppLogger.php`).
3. The server stores the raw message and registers which **company WhatsApp number** received/sent it.
4. The server tries to create a matching **Action** in the CRM when the counterparty phone matches a Contact.
5. Staff can **browse logs** in the Logger UI and **backfill missed Actions** from the Actions app settings.

Nothing is permanently stored in the extension except a short local popup log (last ~100 entries) for debugging. All business data lives in the database.

---

## 2. Components and locations

| Component | Folder / file | Role |
|-----------|---------------|------|
| Chrome extension | `whatsapploggerextension/` | Captures WhatsApp Web messages and POSTs them to the API |
| Logger API | `crmapi/WhatsAppLogger.php` | Receives messages, stores logs, exposes list/get/stats |
| Shared helpers | `crmapi/global.php` | Phone normalization, contact lookup, account registration, action sync |
| Actions API | `crmapi/Actions.php` | CRM actions module; includes `syncwhatsapplogger` backfill command |
| Logger UI | `whatsapploggerui/` | Read-only web app to browse stored messages |
| Actions UI | `actionsui/` | CRM actions app; settings button to run WhatsApp backfill |

Database tables (in `asystircrm`):

- **`WhatsAppAccounts`** — company receiver phones per tenant (`accountid`). One row per distinct number linked on a machine.
- **`WhatsAppLog`** — every captured message, deduplicated by `(accountid, messageid)`.

---

## 3. High-level architecture

```
┌──────────────────────────────────────────────────────────────────────────┐
│                         Chrome (user machine)                            │
│                                                                          │
│  web.whatsapp.com                                                        │
│       │                                                                  │
│       ▼                                                                  │
│  content.js          MutationObserver on message DOM                     │
│       │              Extracts message_id, text, direction, phone, etc.   │
│       │                                                                  │
│       ▼                                                                  │
│  background.js       Adds apikey, receiver_phone, command=logmessage     │
│       │              POST JSON → crmapi/WhatsAppLogger.php             │
│       │              Retries on failure (chrome.alarms)                  │
│       ▼                                                                  │
│  popup.html/js       Config: API URL, API key, company receiver phone    │
└──────────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌──────────────────────────────────────────────────────────────────────────┐
│                         crmapi (server)                                  │
│                                                                          │
│  WhatsAppLogger.php  command=logmessage                                  │
│       │              1. Validate API key (tenant)                          │
│       │              2. ensurewhatsappaccount(receiver_phone)            │
│       │              3. Insert WhatsAppLog (skip if message_id exists)   │
│       │              4. syncwhatsapplogtoaction() → Actions table          │
│       │                                                                  │
│  global.php          normalizephone, findcontactbyphone,               │
│                      ensurewhatsappaccount, syncwhatsapplogtoaction      │
│                                                                          │
│  Actions.php         syncwhatsapplogger (60-day backfill on demand)      │
└──────────────────────────────────────────────────────────────────────────┘
          │                                    │
          ▼                                    ▼
┌─────────────────────┐              ┌─────────────────────────┐
│  whatsapploggerui   │              │  actionsui               │
│  Browse WhatsAppLog │              │  View WhatsApp actions   │
│  Filter by phone,   │              │  Settings → Check on     │
│  date, direction    │              │  WhatsApp Logger         │
└─────────────────────┘              └─────────────────────────┘
```

---

## 4. Message flow (step by step)

### 4.1 Capture (extension)

1. User opens **https://web.whatsapp.com** with the extension enabled.
2. `content.js` watches the DOM for new message nodes (`[data-id]`).
3. For each new message it builds a payload:

   | Extension field | Meaning |
   |-----------------|---------|
   | `message_id` | WhatsApp `data-id` (unique per message) |
   | `phone_number` | Counterparty phone or group JID from JID parsing |
   | `chat_name` | Header display name |
   | `chat_type` | `individual` or `group` |
   | `sender` / `sender_phone` | Who sent it |
   | `direction` | `incoming` or `outgoing` |
   | `text` | Message body |
   | `wa_timestamp` | ISO timestamp |
   | `source` | `whatsapp_web` |

4. In-memory dedup: if `message_id` was already sent this session, skip.
5. Payload is sent to `background.js` via `chrome.runtime.sendMessage`.

### 4.2 Delivery (background service worker)

1. Reads saved settings from `chrome.storage.sync`:
   - `loggerUrl` (default: `http://localhost/WebSite/crmapi/WhatsAppLogger.php`)
   - `apiKey` (**required**)
   - `receiverPhone` (**required**) — the logged-in company WhatsApp number
   - `receiverLabel` (optional) — friendly name, e.g. "Sales line"
2. Merges into POST body:
   - `command: logmessage`
   - `apikey` + header `X-API-Key`
   - `receiver_phone`, `receiver_label`
3. POSTs JSON to the Logger API.
4. On failure: retries up to 3 times with exponential backoff; then marks entry as failed in local popup log.

### 4.3 Ingest (WhatsAppLogger.php)

1. **`IsValidToken()`** — validates API key and resolves `accountid` (tenant).
2. **`ensurewhatsappaccount()`** — normalizes `receiver_phone` and:
   - If that number **already exists** for the tenant → update `lastseen` (and label if provided).
   - If it is a **new number** → **insert** a new `WhatsAppAccounts` row.
   
   This covers the case where someone relinks a different company number on the same Chrome profile: each distinct number gets its own account record automatically.

3. **Duplicate check** — if `(accountid, messageid)` already exists → return success with `duplicate: true` (no second row).
4. **Insert `WhatsAppLog`** with counterparty phone, chat metadata, text, timestamps, `actionsynced = false`.
5. **Call `syncwhatsapplogtoaction()`** immediately (see below).

### 4.4 Action sync (global.php → Actions table)

Runs on every new log row (and again during manual backfill):

1. Skip if already synced with an action id.
2. Skip **group chats** (`chattype = group`) → mark `actionsynced = skipped`.
3. **`findcontactbyphone()`** on CRM `Contacts` (`phonemobile`, `phonework`, `phonehome`).
4. If **no contact** → mark `actionsynced = skipped`, reason `no_contact_match`.
5. If contact found → set `clientid` / `leadid` / `targetid` from contact `typename` (`client`, `lead`, `vendor`).
6. Skip if an Action with the same `externalref = messageid` already exists.
7. Insert Action: `type = whatsapp`, `source = whatsapp_logger`, HTML body with message text.
8. Update log row: `actionsynced = true`, `actionid`, `actionsyncedat`.

**Important:** Messages from unknown numbers are still stored in `WhatsAppLog`; they simply do not create Actions.

---

## 5. Deduplication (two layers)

| Layer | Where | How |
|-------|-------|-----|
| 1 — Browser session | `content.js` in-memory `Set` | Avoids duplicate POSTs while tab is open |
| 2 — Database | `WhatsAppLog` unique key `(accountid, messageid)` | Permanent dedup across sessions and machines |

WhatsApp’s `data-id` is stable; the backend treats it as the canonical message identifier.

---

## 6. Company phone registration

Every POST includes **`receiver_phone`** (the company’s WhatsApp number for that browser).

- First message with `5491110000001` → creates `WhatsAppAccounts` row.
- Later messages with the same number → updates `lastseen` only.
- User changes popup to `5491110000002` → **new** `WhatsAppAccounts` row; logs are tied to `whatsappaccountid`.

The Logger UI phone filter lists all registered company numbers for the tenant.

---

## 7. Logger UI (whatsapploggerui)

Read-only browser app. Typical URL:

`http://localhost/WebSite/whatsapploggerui/WhatsAppLogger.html`

1. User enters CRM **API key** (same as extension).
2. Loads company phones via `command=listphones`.
3. Lists messages via `command=showlist` with filters:
   - Company phone (`whatsappaccountid`)
   - Date range
   - Direction (incoming / outgoing)
   - Text search
4. Click a row → `command=get` for full detail (no edit/delete).

Also available: `command=stats` for synced / pending / skipped counts.

---

## 8. Actions UI and backfill

When the Actions API was temporarily unavailable, messages may be stored in `WhatsAppLog` with `actionsynced = false` but no Action created.

**Recovery:**

1. Open **actionsui** → ⚙ Settings.
2. Click **“Check on WhatsApp Logger”**.
3. Calls `Actions.php` with `command=syncwhatsapplogger` and `days=60`.
4. Server selects all logs from the last 60 days where `actionsynced` is false/empty and runs `syncwhatsapplogtoaction()` on each.

Response includes: `total`, `synced`, `skipped`, `errors`.

Use this after outages, schema fixes, or after adding new Contacts whose phones now match older messages (note: rows already marked `skipped` are not retried unless you clear that status in the database).

---

## 9. Authentication

All API calls use the CRM **API key**:

- Extension: stored in popup, sent as JSON `apikey` and header `X-API-Key`.
- Logger UI / Actions UI: API key from login field / local storage, same header.

The key resolves to an `accountid` (tenant). All queries are scoped to that tenant.

---

## 10. API commands reference

### WhatsAppLogger.php

| Command | Used by | Purpose |
|---------|---------|---------|
| `logmessage` | Extension | Store one message + trigger action sync |
| `showlist` / `list` | Logger UI | Paginated log list with filters |
| `get` | Logger UI | Single log record |
| `listphones` | Logger UI | Company WhatsApp numbers for filter |
| `stats` | Logger UI | Sync counts |

First visit to the module URL may run **setup** and create tables if they do not exist.

### Actions.php

| Command | Used by | Purpose |
|---------|---------|---------|
| `syncwhatsapplogger` | Actions UI settings | Backfill Actions from unsynced logs (default 60 days) |

Optional POST parameter: `days` (1–365, default 60).

---

## 11. Configuration checklist

| Setting | Where | Notes |
|---------|-------|-------|
| API key | Extension popup, UIs | Must be valid in `crmapi` ApiKeys table |
| Logger URL | Extension popup | Must reach your server’s `WhatsAppLogger.php` |
| Receiver phone | Extension popup | Digits only, country code included (e.g. `5491112345678`) |
| Receiver label | Extension popup | Optional display name |
| `$WhatsAppLogger = 'yes'` | `crmapi/Config.php` | Enables module |
| Database tables | `WhatsAppAccounts`, `WhatsAppLog` | See `crmapi/innerdoc/whatsapplogger_schema.sql` |

---

## 12. Failure modes

| Symptom | Likely cause |
|---------|----------------|
| Popup shows failed sends | Wrong URL, API key, or server down; check browser console / Apache logs |
| “API key not configured” | Save API key in extension popup |
| “Company receiver phone not configured” | Save receiver phone in popup |
| Logs appear but no Actions | Counterparty not in Contacts, or chat is a group |
| Duplicate logs in DB | Should not happen; check unique index on `(accountid, messageid)` |
| Old messages never get Actions | Run **Check on WhatsApp Logger** in Actions settings; ensure contact exists |

---

## 13. Related files

| Document / path | Content |
|-----------------|---------|
| [`SPEC.md`](SPEC.md) | Original full product specification |
| [`CHROME_MANUAL_INSTALL.md`](CHROME_MANUAL_INSTALL.md) | Load unpacked extension in Chrome |
| `whatsapploggerextension/content.js` | DOM capture logic |
| `whatsapploggerextension/background.js` | HTTP POST + retries |
| `crmapi/WhatsAppLogger.php` | Logger API module |
| `crmapi/global.php` | Phone + sync helpers |
| `crmapi/Actions.php` | Backfill command |
| `whatsapploggerui/` | Logger browser UI |