Smtp Server

Let’s start with a brief explanation of what SMTP is in a nutshell.

“The Simple Mail Transfer Protocol (SMTP) is an internet standard communication protocol for electronic mail transmission. Mail servers and other message transfer agents use SMTP to send and receive mail messages.” Wikipedia

I think that definition sums it up nicely. You could think of SMTP as the language transfer agents use to communicate with each other to safely deliver your email to it’s recipient. As with most internet communication these agents consist of mail clients and servers.

Our goal in this blog post is to demonstrate SMTP in action, by creating a simple SMTP client and server, and getting the two talking.

• how to make use of JavaScript Set’s for maintaining whitelists
• how to spin up a client in one line of self evaluating code directly from your terminal
• how to accept user input via process.stdin, and process it one line at a time, using the Readline module
• how distinguish between ctrl+c and ctrl+d end of input transmission signals with respect to the Readline ‘close’ event
• how to use stream.pipeline to safely send data from one stream to another.

SMTP server

Creation of our SMTP server goes something like:

• set up folder structure for user mailboxes where messages will be stored
• create user and host email recipient whitelists
• register an event listener for accepting or rejecting incoming mail requests
• register an event listener for saving email messages to mailboxes
• create server (instance of Node’s net.Server under the hood) using smtp-protocol
• spin up server to listen for SMTP beeps and boops

// declare whitelists
const hosts = new Set(["hostmeister.de", "hostgator.com"]);
const userMailboxes = new Set(["user1", "user2"]);
const mailDir = path.join(__dirname, "mail");

// create mailboxes if not exist
function initMailboxes() {
  function makeDirs(dir) {
    try {
      return fs.mkdirSync(dir, { recursive: true });
    } catch (err) {
      if (err.code !== "EEXIST") throw err;
      else console.log("mailbox exists, skipping creation...");
    }
  }
  for (let usrDir of userMailboxes) makeDirs(`${mailDir}/${usrDir}`);
}

We start off by declaring the host and user whitelists. Emails to recipients whose email provider is not in this list would not be accepted.

initMailboxes() uses the file system module’s mkdirSync method to traverse our user mailboxes, and create the mailbox folder structure. Passing the recursive option allows for creating nested structures even if they don’t exist yet.

The next chunk of code contains the main server logic.

const server = smtp.createServer((req) => {
  req
    .on("to", (to, ack) => {
      console.log("TO event received");
      const [user, host] = to.split("@");
      if (hosts.has(host) && userMailboxes.has(user)) ack.accept();
      else ack.reject();
    })
    .on("message", (stream, ack) => {
      console.log("MSG event received");
      const { from, to } = req;
      ack.accept(250, "hello");
      to.forEach((recipient) => {
        const [user] = recipient.split("@");
        const dest = path.join(mailDir, user, `${from}-${Date.now()}`);
        const mail = fs.createWriteStream(dest);
        mail.on("end", () => console.log("ending"));
        mail.write(`From: ${from} \n`);
        mail.write(`To: ${recipient} \n\n`);
        stream.pipe(mail);
      });
    })
    .on("error", (err) => console.error(err));
});

smtp.createServer() creates a new net.Server, and fires the callback function we pass as first argument, for every new connection. We use the req EventEmitter it exposes to us, to register functionality for specific event types.

The first event we listen for is the ’to’ event. This event is emitted when the ‘RCPT TO:’ command is received by the server. The To event listener has 2 parameters (foof!) The first (to) gives us whatever was entered after the ‘RCPT TO:’ command. The second (ack) contains functions for accepting or rejecting recipients. We’re using this opportunity to run some checks on the recipient email inputted. We split the email into user and host, and check whether each entity is in our whitelists. We initialised our user and host whitelists as JavaScript Sets, to leverage the handy Has method, and accept or reject recipient emails accordingly. Calling accept() on a recipient pushes the recipient to an array of recipients that the server uses later on when saving messages.

The other event we’re listening for is ‘message’. This is where we save incoming, accepted messages to our user mailbox/s. The message event in emitted when the ‘DATA’ command is received by the server. At this stage we deconstruct req.from and req.to properties from our req event emitter, and immediately call accept since we’re not doing any validation on message content itself. req.to is an array of accepted recipients, therefore we forEach() over every recipient, and save each message to the respective user’s mailbox.

We construct our message by creating a write stream for each user, writing a couple of lines manually using stream.write(), and piping the message contents from stream (a readable stream which is given to us by the message event listener) to our target write stream.

That’s pretty much it for the server part. Here’s a small run through showcasing what the above would look like.

initMailboxes();
console.log("running server");
server.listen(25);

We’ll run the server in one terminal, using net.Server.listen() on port 25.

node -e "process.stdin.pipe(require('net').connect(25)).pipe(process.stdout)"

We also spin up a pseudo client for the demo in a second terminal. We do this by evaluating a string of node code, which we supply as an argument using Node’s -e evaluation flag. Here, we listen for user input via stdin, and route this input to a socket, which is connected to our server. net.connect() creates a new net.Socket, connects it to localhost on port 25 (port 25 is considered to be the default transmission channel for sending email), and returns a reference to our newly connected socket. This socket is simply an abstraction of a TCP connection, and a means by which we can communicate directly with the server.

net.Socket extends stream.Duplex, which means we can use stream.pipe to channel the server’s responses back to our terminal’s display via stdout. The socket is also an EventEmitter, meaning we could attach listeners to noteworthy events should we need to.

SMTP Client

Our nifty smtp-protocol module also has some functionality we can use to build a simple SMTP client to communicate with our server like we just did above.

Client will work as follows:

• provide a means by which we can accept user input
• create a connection to server
• greet the server with client hostname
• set the from address
• prompt for a list of recipients
• prompt for email body
• accept end of input
• send messages to server

Let’s go.

const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});

We begin by creating a means by which we can accept user input, namely, readline.createInterface() which returns a readline.Interface. It is aptly named since it acts as an interface between a readable stream, in our case user input from the terminal via process.stdin, and a write stream, process.stdout to broadcast output to the user.

The next block of code contains everything else we need.

smtp.connect(25, (client) => {
  let body = "";
  client.helo(os.hostname());
  client.from("info@suitehearts.net");
  rl.question("To: ", (recipients) => {
    recipients
      .split(/;|,/gm)
      .forEach((recipient) => client.to(recipient.trim()));
    rl.write("(^C to cancel) | (^D to send)\n");
    rl.write("Enter email message: \n");
    client.data((err, code, lines) => {
      console.log(err, code, lines);
    });
    rl.on("line", (line) => (body = body.concat(line, "\r\n")));
    rl.on("close", () => {
      console.log("sending message/s");
      stream.pipeline(bodyReadable(body), serverWritable(client), (err) => {
        if (err) console.error("something went wrong holmes...");
        client.quit();
      });
    }); // close event is emitted
  });
});

function bodyReadable(msgBody) {
  const streamMsg = new stream.Readable();
  streamMsg.push(msgBody);
  streamMsg.push(null);
  return streamMsg;
}

function serverWritable(client) {
  const serverStream = client.message((err, code, lines) => {
    console.log(err, code, lines);
    // handle response paths
  });
  return serverStream;
}

Let’s unpack this big blob of code.

The smtp-module has a .connect() method which is a wrapper around Node’s net.createConnection() and essentially creates a new SMTP client connection, defaulting to localhost. We register a callback function which fires for each new connection to the server. This gives us a reference to Client which will do all the talking.

We greet the server using .helo, set the from address, and prompt for user input using rl.question() This command prompts for user input, where we accept a ‘;/,’ separated list of recipients, and invokes the callback function with the user input (ie our recipients) as the first argument to the callback. We split the list, and iterate over each recipient, calling our Client’s .to() which passes this information to our server, just like we did with the ‘mail from:’ input in the server example. (you could pass a callback function along with the recipients which could be used to intercept server response for error handling)

Once we’ve set the recipients, the next thing we do is instruct the server that we are about to send the message data using Client.Data(), and prompt the user to enter the message body. We’re using the ’line’ event to capture every line of input. This event is fired whenever the user hits enter. As with all events, we have a listener which is called with each line input, appending it to our final message body.

We now need a way of telling the client that we’re done with the message and would like to proceed with sending. This could be done by registering a listener to the ‘close’ event. So ‘close’ will essentially be used to send. The only problem is that the following actions will all cause close to be emitted:

• Explicitly calling rl.close() (which we’re not doing so all good)
• Signalling end of transmission with Ctrl+D
• Signalling SIGINT with Ctrl+C

That means that should the user want to abandon the process by signalling Ctrl+C, this would call our close listener which is not what we want. Enter SIGINT.

rl.on("SIGINT", () => {
  console.log("... cancelled ...");
  process.exit();
});

By registering a listener for the SIGINT event, we will prevent it from emitting close and help us to provide different courses of action for Ctrl+D (in which case we send), and Ctrl+C in which we abandon the process altogether.

We finally get to send our messages.

In our ‘close’ listener, we:

• create a new readable stream, push our message data to it (this is done in the bodyReadable helper function)
• call Client.message() which returns us a writable stream that we can use to send data to server
• we wrap our readable and writable streams from points 1 and 2 in a stream.Pipeline which essentially pipes our message from client to server.

We could have also streamed our message to the server like bodyReadable.pipe(serverWritable) however Node does not propagate errors through each stream, and you would need to handle errors and release resources for each stream manually in case of failures.

Here’s the client in action:

That’s it. Hope you enjoyed.

Feedback welcome: info AT suitehearts DOT net.