(The following examples show the JSON representation of KISSmetrics events)
{"_n": "revenue event", "amount": "10", "_p": "user@example.org", "_t":1333266186}
{"_n": "NOT a revenue event", "amount": "10", "_p": "user@example.org", "_t":1333266186}

The thing to note is that even though only the first line should be considered a revenue event, KISSmetrics only takes the “amount” property into consideration and assumes the second line is also a revenue related event. This is kinda stupid and non-intuitive, and I don’t think it was clear in the documentation.

This caused KISSmetrics to report revenue at more than double the actual amount. We didn’t realize the data was so wrong for over a month, so rather than just deleting all the data and start over, I investigated (wrote KISSmetrics support) how to fix this without losing all our historic data.

The trick for how to fix this is to realize that to get rid of all the bad data we can change the names of the events and properties and tell KISSmetrics to start considering the new event/property names for tracking revenue.

The first step is to modify our code that generates our real revenue event to use different event and property names. So now our revenue event looks like this:

{"_n": "revenue event new", "amount new": "10", "_p": "user@example.org", "_t":1333266186}

We can just leave the non revenue event alone.

Once you are submitting data to KISSmetrics using only these new names for your real revenue event, you can go into KISSmetrics settings to change the revenue tracking event and property. Here are the steps for doing this:

1. Click ‘Metrics’
2. Click on one of the ‘Revenue’ related metric.
3. Click ‘Edit metric details’
4. Change the ‘Property’ drop down to ‘amount new’ (for the example above, or whatever the new revenue property is.)
5. Click ‘Save metric’.

Repeat the above for any other metrics that track revenue related events.

We also need to change the revenue event:

1. Click the Settings “gear” in the top menu.
2. Under ‘Event Library’ click ‘set mappings’.
3. Click on the ‘Purchased’ mapping.
4. Change the drop down to ‘Revenue event new’ (for the above example, or whatever the name of your new revenue event is.)
5. Click ‘Save changes’.
6. This will prompt you to update the funnels that use this event, check all the boxes and click ‘Save changes’ again.

Repeated the above for the ‘Revenue’ mapping. Changing it to the ‘amount new’ event.

So now we have KISSmetrics using the new data, but we’ve lost all our historic data. To get that back we need to:

1. Export the old data.
2. Filter out just the old revenue event.
3. Rewrite it to use the new revenue event name and property.
4. Import the altered data to give us our historic revenue data back.

To export your data from KISSmetrics you need an Amazon Web Services account. I will assume you have one and know how to use it to create S3 buckets and download data from them.

To configure KISSmetrics to use your AWS account, follow these instructions.

Then you wait for KISSmetrics to finish exporting your data. I had less than 2 months of data and it took maybe an hour for it to all show up in my AWS bucket. I then used s3cmd to download the contents of the bucket to my development machine.

At this point you have a ton of options for how you want to process the data. I’m currently taking an online course (PL101: Create Your Own Programming Language) that uses Javascript/Node.js so I am going to do this with Node.js.

I used the following script to find my revenue events and convert them to use the new event and property names:

var fs = require('fs');

var eventNameOld = "revenue event";
var eventNameNew = "revenue event new";
var propertyNameOld = "amount";
var propertyNameNew = "amount new";

var processEvent = function(event) {
    if (event._n === eventNameOld) {
        event._n = eventNameNew;
        // _d = 1 is specified to make KISSmetrics use the existing _t timestamp
        // instead of using the time the server received this event
        event._d = 1;
        event[propertyNameNew] = event[propertyNameOld];
        delete event[propertyNameOld];

        console.log(JSON.stringify(event));
    }
}

var processLine = function(line, index, array) {
    // JSON doesn't like trying to parse an empty line, understandable
    if (line.length === 0) { return; }
    // KISSmetrics has some data that fails to parse, especially in referral strings
    // For my purposes I'm assuming these failed lines are not revenue events
    // but you should examine the output of STDERR to see if you need any of these lines
    try {
        processEvent(JSON.parse(line));
    } catch(error) {
        console.warn("Got error parsing: " + line + " Error: " + error);
    }
}

process.argv.slice(2).forEach(function (val, index, array) {
    var fileContents = fs.readFileSync(val, 'utf-8');
    var lines = fileContents.split('\n');
    lines.forEach(processLine);
});

And I run it like so: node convert-revenue.js 2012/*/*.json > revenue-events.json

Now I want to take the contents of revenue-events.json and import it back into KISSmetrics. I used the following script:

var fs = require('fs');
var https = require('https');

var kissMetricsAPIKey = "YOUR API KEY GOES HERE";

var createRequest = function(str) {
    return options = {
        host: "trk.kissmetrics.com",
        path: "/e?" + str + "&_k=" + kissMetricsAPIKey
    };
}

var serialize = function(obj) {
    var properties = [];
    for (var p in obj) {
        properties.push(encodeURIComponent(p) + "=" + encodeURIComponent(obj[p]));
    }
    return properties.join("&");
}

var processEvent = function(event) {
    return createRequest(serialize(event));
}

var processLine = function(line) {
    // JSON doesn't like trying to parse an empty line, understandable
    if (line.length === 0) { return; }
    // KISSmetrics has some data that fails to parse, especially in referral strings
    // For my purposes I'm assuming these failed lines are not revenue events
    // but you should examine the output of STDERR to see if you need any of these lines
    try {
        return processEvent(JSON.parse(line));
    } catch(error) {
        console.warn("Got error parsing: " + line + " Error: " + error);
    }
}

process.argv.slice(2).forEach(function (val, index, array) {
    var fileContents = fs.readFileSync(val, 'utf-8');
    var lines = fileContents.split('\n');
    var requests = [];
    for (var i = 0; i < lines.length; i++) {
       if (lines[i] != "") {
           requests.push(processLine(lines[i]));
       }
    }

    var go = function() {
        if (requests.length > 0) {
            var request = requests.pop();
            https.get(request, function(res) {
                if (res.statusCode != 200) {
                    console.error("Got status: " + res.statusCode + " retrying.");
                    requests.push(request);
                }
                res.on('end', function() {
                    console.log("Finished request: " + request.path);
                    go();
                });
            }).on('error', function(e) {
                console.error("Got error " + e + " for request: " + request.path + " retrying.");
                requests.push(request);
                go();
            });
        } else {
            console.log("All done.");
        }
    };
    go();
});

Run it with node import-kissmetrics.js revenue-events.json

It took me a while to come up with the go method which allowed me to serialize the http requests so I wasn’t overloading KISSmetric. Also, I’m assuming the retry logic works, but I haven’t had any errors submitting data yet so this is untested.

So far it looks like the issue is fixed. It took me a few days to work out all the places where the old property and event were used. Once I had all of them using the new names, the data fell in line.