# Email Logger — System overview

How the Email Logger components work together: cron IMAP polling, crmapi, Logger UI, and Actions module.

- **Backend detail:** [`../../crmapi/innerdoc/EMAIL_LOGGER_SPEC.md`](../../crmapi/innerdoc/EMAIL_LOGGER_SPEC.md)
- **UI detail:** [`SPEC.md`](SPEC.md)
- **UI mockups:** [`ui/emaillogger_inbox_view.png`](ui/emaillogger_inbox_view.png), [`ui/emaillogger_conversation_view.png`](ui/emaillogger_conversation_view.png)

---

## 1. What the system does

1. An admin configures company **IMAP mailboxes** in emailloggerui settings.
2. A **cron job every 30 minutes** calls `EmailLogger.php` → `runfetch`.
3. The backend connects to each mailbox **read-only**, downloads new messages, stores them in `EmailLog`.
4. For each new message, the server tries to create a CRM **Action** when the counterparty email matches a Contact.
5. Staff browse mail in **emailloggerui** (inbox + conversation views).
6. If Actions sync was missed, staff run **Check on Email Logger** from actionsui settings (60-day backfill).

Mail on the IMAP server is **never deleted** by this system.

---

## 2. Components

| Component | Location | Role |
|-----------|----------|------|
| Cron | Server crontab | Triggers `runfetch` every 30 min |
| Logger API | `crmapi/EmailLogger.php` | IMAP poll, store logs, CRUD mailboxes, list APIs |
| CLI wrapper | `crmapi/EmailLoggerCron.php` | Secure cron entry (planned) |
| Shared helpers | `crmapi/global.php` | Email normalize, contact lookup, action sync |
| Actions API | `crmapi/Actions.php` | `syncemaillogger` backfill command |
| Logger UI | `emailloggerui/` | Inbox, conversations, mailbox settings |
| Actions UI | `actionsui/` | “Check on Email Logger” in settings |

### Database tables

| Table | Purpose |
|-------|---------|
| `EmailAccounts` | Company mailbox + IMAP credentials (password encrypted) |
| `EmailAccountFolderState` | Last UID synced per IMAP folder |
| `EmailLog` | Every captured email; dedup by Message-ID and IMAP UID |

---

## 3. Architecture diagram

```
┌──────────────────────────────────────────────────────────────────────────┐
│  Server cron (*/30 * * * *)                                              │
│  php EmailLoggerCron.php  →  command=runfetch  +  cronkey                 │
└──────────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌──────────────────────────────────────────────────────────────────────────┐
│  crmapi/EmailLogger.php                                                  │
│  For each active EmailAccounts:                                          │
│    IMAP connect (read-only) → INBOX, Sent, Drafts [+ Outbox]             │
│    New UIDs → parse → EmailLog INSERT → syncemaillogtoaction()             │
└──────────────────────────────────────────────────────────────────────────┘
          │                              │
          ▼                              ▼
   Company IMAP server              MySQL (EmailLog)
   (mail unchanged)                      │
                                        ▼
                              Actions (type=email)
                                        ▲
                                        │
┌──────────────────────┐     ┌─────────────────────────┐
│  emailloggerui       │     │  actionsui               │
│  Browse / search     │     │  Check on Email Logger   │
│  Mailbox settings    │     │  syncemaillogger 60d     │
│  Refresh inbox       │     └─────────────────────────┘
└──────────────────────┘
```

---

## 4. Message flow (step by step)

### 4.1 Setup (one-time per mailbox)

1. User opens emailloggerui → Settings.
2. Adds mailbox: address, IMAP host/port, credentials, folder names.
3. **Test connection** validates IMAP.
4. Row saved in `EmailAccounts`.

### 4.2 Cron fetch

1. Cron runs `runfetch`.
2. Load all `EmailAccounts` where `active = true`.
3. For each account and folder (inbox, sent, drafts, …):
   - Decrypt password, open IMAP (SSL/TLS).
   - Read `EmailAccountFolderState.lastuid`.
   - Fetch messages with UID > lastuid (peek, no delete).
   - Parse headers and body.
   - Determine `direction`, `folder`, `counterpartyemail`.
   - Insert `EmailLog` if not duplicate.
   - Call `syncemaillogtoaction()`.
   - Update folder `lastuid`.
4. Update account `lastchecked`, `lastfetchstatus`.

### 4.3 Counterparty and direction

| Source folder | direction | counterpartyemail |
|---------------|-----------|-------------------|
| Inbox | incoming | From address |
| Sent | outgoing | First To address |
| Drafts | outgoing | First To address (if any) |

Used for CRM contact match and conversation grouping.

### 4.4 Action sync (automatic)

`syncemaillogtoaction()` in `global.php`:

1. Skip if already synced.
2. Skip if no counterparty email.
3. `findcontactbyemail()` on Contacts (`email`, `email2`, `emails` JSON).
4. No match → `actionsynced = skipped`, reason `no_contact_match`.
5. Match → insert Action (`type=email`, `source=email_logger`), link `clientid`/`leadid`/`targetid`.
6. Update `EmailLog.actionid`, `actionsynced = true`.

Unknown senders still appear in emailloggerui; they just do not create Actions.

### 4.5 Manual refresh

User clicks **Refresh inbox** in emailloggerui → `fetchaccount` for selected mailbox → same fetch logic as cron for one account.

---

## 5. Deduplication

| Layer | Mechanism |
|-------|-----------|
| Message-ID | Unique `(accountid, messageid)` when header present |
| IMAP UID | Unique `(accountid, emailaccountid, imapfolder, imapuid)` |

Re-fetching the same UID or Message-ID returns duplicate success without second row.

---

## 6. UI views

### 6.1 Folder views (Inbox, Sent, Drafts, Outbox)

Classic email client: list in center, preview on right.

API: `showlist` with `folder=inbox|sent|drafts|outbox`.

See mockup: [`ui/emaillogger_inbox_view.png`](ui/emaillogger_inbox_view.png)

### 6.2 Conversations view

Groups all mail by `counterpartyemail` — like a chat app contact list.

- Center: one row per counterparty (name, last subject, date).
- Right: chronological thread (`getconversation`).
- Show CRM badge (CLIENT, etc.) when contact matched.

See mockup: [`ui/emaillogger_conversation_view.png`](ui/emaillogger_conversation_view.png)

### 6.3 Compose

**Compose email** opens the user’s PC mail client (`mailto:`) or optional web compose URL — **not** an in-app composer.

---

## 7. Authentication

- emailloggerui, actionsui: CRM **API key** (`X-API-Key` header).
- Cron: `$EmailLoggerCronKey` in Config.php (separate secret).

All data scoped by tenant `accountid`.

---

## 8. Actions backfill

When Actions.php was unavailable, messages may sit in `EmailLog` with `actionsynced = false`.

**Recovery:**

1. actionsui → ⚙ Settings.
2. **Check on Email Logger** (alongside existing WhatsApp button).
3. `Actions.php` → `command=syncemaillogger`, `days=60`.
4. Scans unsynced logs from last 60 days, runs `syncemaillogtoaction()` on each.

Response: `{ status, days, total, synced, skipped, errors }`.

Rows already marked `skipped` (no contact) are not retried unless reset in DB after adding the contact.

### Actions UI specification (future implementation)

In [`actionsui/Actions.html`](../../actionsui/Actions.html) settings panel, WhatsApp section pattern:

```html
<div class="SettingsSection SettingsSectionWide">
  <div class="SettingsSectionHeader"><div>Email Logger</div></div>
  <div class="EmailLoggerSyncPanel">
    <div class="SmallText">Scan the last 60 days of email logs and register missed Actions.</div>
    <div onclick="actions.checkemaillogger()" class="Button SecondaryButton">Check on Email Logger</div>
    <div class="EmailLoggerSyncStatus SmallText"></div>
  </div>
</div>
```

In [`actionsui/Actions.js`](../../actionsui/Actions.js):

```javascript
function checkemaillogger() {
	var obj = {};
	obj.command = 'syncemaillogger';
	obj.days = 60;
	apipost('Actions.php', obj, function (data) { /* toast with results */ });
}
```

Backend: [`syncwhatsapplogger()`](../crmapi/Actions.php) parallel → `syncemaillogger()` using `EmailLog` table.

---

## 9. Comparison with WhatsApp Logger

| Aspect | WhatsApp | Email |
|--------|----------|-------|
| Capture | Chrome extension | Cron + IMAP |
| Account identity | `receiver_phone` per message | Pre-configured `EmailAccounts` |
| Account auto-register | New phone → new row on each POST | Admin adds mailbox in settings |
| Dedup key | WhatsApp `data-id` | Message-ID + IMAP UID |
| Contact match | Phone fields | Email fields |
| Logger UI | List + detail | Inbox + **conversations** |
| Compose | N/A | External mail client only |
| Server side effect | None on WhatsApp | **Never delete IMAP mail** |
| Backfill | `syncwhatsapplogger` | `syncemaillogger` |

---

## 10. Configuration checklist

| Item | Where |
|------|-------|
| `$EmailLogger = 'yes'` | `crmapi/Config.php` |
| `$EmailLoggerCronKey` | `crmapi/Config.php` |
| PHP `imap` extension | Server |
| Cron `*/30 * * * *` | crontab |
| Tables | Run `emaillogger_schema.sql` |
| Mailboxes | emailloggerui settings |

---

## 11. Out of scope (v1)

- OAuth2 for Gmail / Microsoft 365
- In-app compose or send
- Attachment storage
- POP3
- Delete/move on IMAP server

---

## 12. Related files

| Path | Description |
|------|-------------|
| [`../../crmapi/innerdoc/EMAIL_LOGGER_SPEC.md`](../../crmapi/innerdoc/EMAIL_LOGGER_SPEC.md) | Full backend spec |
| [`../../crmapi/innerdoc/emaillogger_schema.sql`](../../crmapi/innerdoc/emaillogger_schema.sql) | DDL |
| [`SPEC.md`](SPEC.md) | UI specification |
| [`../../whatsapploggerextension/innerdocs/SYSTEM_OVERVIEW.md`](../../whatsapploggerextension/innerdocs/SYSTEM_OVERVIEW.md) | WhatsApp equivalent |