Previous Entry В избранное Поделиться Next Entry
Serverside JavaScript, 1ms for transform.
andrewsumin

What for?


“What for?” — that is the key question in any decision. In our case we had several answers to this question.


First – developers. Current template engine was written in C# and making any changes in it required time. The key reason – this template engine was developed by certain people, while used by others.


To my mind, it’s quite a common practice among coders. There is no doubt that coders need tools, but these tools are implemented by people who hardly have any idea of day-to-day tasks we face. I’d even say the approach is often like “let them write conditions and cycles, for coding it’s enough”. Maybe that’s the fault of coders, their skills and experience.

Mail.Ru Group has a team of highly qualified specialists, experts in JS, who are able to write a tool and use this tool.


Second - tasks. Let’s take the project mail.ru. We can’t refuse from a template engine on the sever – we need a quick download at first login. We can’t refuse from a template engine on the client – users should see rapid reaction to their actions, which means AJAX and template engine on the client are essential.


The problem is evident: we have two sets of absolutely different templates on the server and on the client. What is most unfair - they solve the same task. Double work simply crocked us up.


V8 is a JavaScript interpreter, and that means we can have one template for both the server and the client.


Third - speed. After a bulk of articles that say many good words about operation time of v8, we decided to check it. But first we were to understand what we want a new template engine to be.



What we need


To begin with I should say that we are restricted by the transformation time on the server, therefore we were not able to implement a highly functional idea. However, we also couldn’t leave the old functions just adding a v8 layer, as that would sound strange.


I’ve been using XSLT transformations for a long time and rather often (not the quickest way of transformation). However, if used correctly, it gives good results (I mean libxslt). XSLT has a very effective template tool – overriding of templates. We decided to realize something similar, but much more simple.


/head.xml
	…
	<title><fest:get name=”title”/></title>
	<fest:set name=”title”>Mail.ru</fest:set>
/mail.xml
	…
	<fest:include src=”head.xml”/>
	<fest:set name=”title”>Почта Mail.ru</fest:set>

That would have been strange if we used v8 and gave no access to JavaScript.


<fest:script>
	var text = “mail.ru”
</fest:script>
<fest:value>text</fest:value>

And many other small things besides standard conditions and cycles.


XML


We used XML as a syntax for a template engine.


Basic functions are provided in IDE. All popular IDE know XML. Tabulation, highlight, basic autocomplete – all plugins you get for free. Validation at IDE level. Valid templates provide valid HTML. Again, all IDE can validate xml.


Extension from the box (name spaces). Very soon you’d want to expand basic features of template engine, eg. add tag to make projects in different languages. Name Spaces makes all these features available.


Wide range of ready-to-use tools. Now we have a great variety of tools for processing XML, eg. libraries to process XML with XSL or a separate class of SAX parcers, XSD and DTD schemes for validation, etc.


Similar syntax of processing constructions and final constructions. In other words, it’s convenient to create XML using XML, fest-templates are easy-to-read. As a plus, data escape is considered.


Realization, cowboy style



So far I was sure that pair programming and testing is the best and most reliable way to complete the task. As usual, tests have proven to remain the best option, meanwhile for pair programming I found alternatives.


The difference of this task from a typical one: we knew a template and final HTML, which was expected to give this template. Kostya (Frontend Developer Mail.Ru Group) and I started to write our realizations. Once a week we compared our transformation speeds.


We followed two different approaches: Kostya compiled a template to a function, and I compiled to a structure. Below you can see an example of the simplest structure:


01	[
02		{action:"template"},
03		"<html>...",
04		{action:"value"},
05		"json.value"
06	]

The second string is the beginning of the template. The third string returns to a browser. The forth string shows that the fifth string is evaled as JavaScript and eval result returns to the browser.


01	[	
02		{action:"template"},
03		"<html>....”,
04		{action:"if"},
05		"json.value",
06		"<span>true</span>",
07		"<span>false</span>"
08	]

More complicated approach. The forth string means that the fifth one is to be evaled, and if the result is true or false, return sixth or seventh string respectively.


There is no need to go into details and explain option with the function.


01	function template(json){
02		var html = "";
03		html += "<html>…";
04		html += json.value;
05		return html;
07	}

Looking ahead I should say Kostya’s approach turned to be faster. For some time we were almost head to head, meanwhile the approach with the structure gave its maximum much earlier.


What this approach gave to us? My first realization completed the task within 200ms. When we took maximum of each of our approaches and combined them, we got 3ms.


In brief, current realization can be described as follows: cycles are compiled to cycles, conditionals to conditionals, etc.


fest:forearch			for(i = 0; i < l; i++) {}
fest:if				if(value) {}
fest:choose			if(value) {} else {}

No narrowing of the scope. Yes, it’s a restriction, but that saves resources. Most important is that narrowing of the scope makes it necessary to take something from a global scope at the level above.


It should be emphasized that templates are compiled to JS-function, which evals in a strict mode. That gives developers no chance to write a code that will provide no memory leak.


JavaScript is available anywhere where data can be logically processed.


<fest:if test=”_javascript_”></fest:if>
<fest:value>_javascript_</fest:value>

All constructions that have evaluation of JavaScript turn to try catch.


All constructions that have HTML return after JavaScript evaluation go through HTML escape on default.


<fest:value>json.name</fest:value>

try {
	html += __escape(json.name)
} catch(e) {}

From the very beginning you can follow up the process of template engine development.
https://github.com/mailru/fest


Integration features/options


On the one hand, v8 is just a library that enables JavaScript interpretation. As a stand-alone tool it seems to be useless – no access to the system. But, on the other hand it can be easily integrated with other programming languages.


Having no experience in C and Perl programming, I made test examples on both languages. Moreover, presently we have integration with Python.


And of course, NodeJS for prototypes and browsers are environments where JavaScript templates work on the fly.


Almost production conditions


When I got 3ms, I went to server-side programmers. To my question “How much time do I have for a request which provides user emails list?” they replied “No more than 4ms”. I already had 3ms for transformation and I decided I should try.


Email list is provided by our own http-server written in C. Data receiving processes do not fight for the processor, therefore we didn’t measure them. We took process of preparing data to transformation and transformation itself.


For some historical reasons our http server stores data in a plain hash.


msg_length = 5
msg_1_title = “letter”
msg_1_Unread = 1

As we are talking about JavaScript, we should first of all think of JSON.


msg = [ {title: “letter”, Unread: true} ]

We took a string with a plain hash, sent it to memory and started to work over the desired result – when JavaScript will operate with JSON at transformation template to v8.


We have tried enormous variants. Throw and object, throw a string and parse it in JavaScript, throw a string and push it through JSON.parse.


That would sound strange, but the quickest option was to transform plain hash to a string that would look the same as JSON and give a string in v8.


“template([ {title: \“letter\”, Unread: true} ])”

However, we got stubborn at 6ms, with 2ms transformation. We were about to give up our idea. Nevertheless, I decided to take initial data, string with a plain hash and, using the same compiled template, get the required HTML in NodeJS.


I got 4ms. When I came to our C programmers with my result, I was expecting to hear something like “That’s cool, but we have no resources to write NodeJS”. But instead I heard “If NodeJS can do that, we also can!”.



At this very moment I realized that we will bring our idea to production. We found a new lease of life!


The solution was simple. If we spend 67% of our time to prepare data, we should ignore this stage. We threw _get(‘key’) function to v8. So, from v8 we took data directly from hash on our http-server. No need to convert data to the required format, no need to transform this string to an object in v8. We got 3ms and had 1ms in store.


Almost production


So, everything looks perfect, but we didn’t even approximately come closer to the production. We feel very anxiety to try.


We take a stand-alone server, run http-server which operates with v8 and duplicate real requests to it. Leave it for 30 hours at one 2.2 GHz Xeon core.


10 000 000+ хитов
1.6ms среднее время трансформации

992 422		10% между	2 и 5ms
208 464		2% между	5 и 10ms
39 649		0,4%		больше 10ms

Only 12% were more than 2ms. V8 runs stabile from memory.


Production


With latest results I came to the Deputy technical director and told that v8 is ready for production. We should launch a small pilot project and if we fail, we could simply forget about it. Deputy technical director replied that the results were good, a stand-alone small project is a good idea, but did I really want to launch v8 to production? He advised to start with Mail.Ru main page. And he was right. We either do serious things in business, or just have fun somewhere aside.


It took 3 days to make main page on fest. We switched one server out of balancer, uploaded a version with v8 there and duplicated the requests. All layouts refer to one daemon/core.


Further on I would like to tell you a very interesting and highly educational story with a happy end. My advise - always study in details what your diagrams show. We transferred half of the load to a test server. The resource consumption of the processor core increased three times, if compared with the usual operation mode. It looked like we failed, and our results were six times worse of compared to current template engine.


We started to analyze. At this stage I will tell you a bit about the architecture of the main page. It contains information from different projects. The information is collected with inner development tool, we call it RB. Out of 165kb generated for the main page, 100kb are collected by RB. Then the process is the following: RB gives parts of HTML through http-server to v8, v8 concats them with its strings and sends the result back to http-server.


Duplicated data throw is evident. We performed optimization. Now v8 gives data directly to http-server, instead of building one big string with data from RB.


__push_string(‘foo’);
__push_rb(id);
__push_string(‘bar’);

As a plus, during concation of strings in v8 no double throw of RB from the server to v8 and back, and most important – any data throw is conversion from utf-8 to utf-16 and back. V8 stores everything in utf-16.


We had a profit. The resource consumption was two times higher than usually, not three as before. Yes, we were still losing by four times, although it seemed we have taken the maximum.


And now the educational part of my story. As an interest I multiplied the load that we tested by two, by the amount of http-servers on the computer and by the number of computers. I got 440 000 000 hits. At this we had 110 000 000 hits per day. Something strange.


We opened logs. It turned out that for each request with the load we received three requests with the log report for statistics! The real load for one http-server was four times lower than the load we tested!


The next morning we deployed version of the main page with v8.


Current situation:
Size of the returned HTML generated by v8 – 65kb.
Time during which v8 processes the request - 1ms.
In average v8 requires 40MB for a scope.


Couple of details


All who deal with v8, should have read an article by Igor Sisoev http://sysoev.ru/prog/v8.html


While we were working with this task, we had a good support from Vyacheslav Egorov - v8 developer (http://mrale.ph).


Our worries about memory were reasonable. If you care about correct actions (providing that lack of memory is a standard situation), you will face problems. You will be able to catch allocation mistake, but all you could do is just perform a correct restart. Frankly speaking, we have only one product where it is critical. As to the main page, we have a substantial memory store and we monitor it carefully.


We discovered the trunk leakage in v8. Vyacheslav failed to reproduce it, but I think we will make a test example which will help the developers to find the leakage. Version 3.6 runs perfect from the memory.


Useful links



https://github.com/mailru/fest fest
https://github.com/mailru/fest v8 API
https://github.com/mailru/fest Igor Sysoev article


Метки: , , ,

Вы читаете andrewsumin