OOXML in the ISO’s limbo

ISO has a press release about the 4 countries (India, Brazil, South Africa and Venezuela) and their appeal against the OOXML standardization.

The important part of it is this:

According to the ISO/IEC rules, a document which is the subject of an appeal cannot be published as an ISO/IEC International Standard while the appeal is going on. Therefore, the decision to publish or not ISO/IEC DIS 29500 as an ISO/IEC International Standard cannot be taken until the outcome of the appeals is known.

Which means OOXML cannot become a standard until the appeals get resolved and all countries get satisfied with the outcome.

In other words, OOXML is stucked in the limbo. It can’t be described, considered or even sold as an ISO standard.

It was sort of approved but can’t be approved. Well, the final specification text was not published yet anyways. With some luck, to technically fix OOXML can take the same amount of time as of solving the appeal.

Then we’ll go back to first question: does the world need tow standard ways to describe “helvetica, 12pt with red” or “a table with 3 columns and 5 rows” or even “page layout with 1.5cm margins” ?

As a reference, Brazilian appeal was published in Andy’s blog and is an informative yet concise enough text to understand the technical and process issues around OOXML.

Iron Man and Programmers

The Iron Man

The Iron Man movie is all about programmers. Being challenged and have to invent something that will make you get rid of hand work, the adventure of a project, to mold, test and debug, and the joy of having a prototype inspiring the next perfect design.

Just don’t miss it. The metal man movie has a lot of Heavy Metal music and Robert Downey Jr., both fit perfectly.

Oh, and stay until after the final credits to see what are the plans for the next movie.

Developers: ISO Standards Do Matter

A post by Michael DeHaan and others inspired me to say a few words about the importance of ISO standards for developers:

Many developers are clamming that good specifications are more important than standards, specially now that the OOXML process opened to the public the dungeons of ISO processes and how the organization produces standards. This is a clear example on how ISO’s reputation is compromised.

But unfortunately “ISO standards” are what governments tend to use. These are the words they put in their Request For Proposals when they are going to buy things. These “ISO standards” are the words they use to claim how they’ll interoperate and trade across borders. And also how public institutions will interoperate with private institutions. Over time, it defines also how private institutions use technology.

So yes, ISO standards for Information Technology do matter from a developer standpoint because they are crucial in shaping the flow of information in the society in general.

Lets put it this way: technical specifications stand for research and development team while to be an ISO standard equals to have a strong sales force. An IT product needs both to succeed.

So you, developer, can use whatever you want or like. But if you want to interoperate with non-developer folks, you — the smartest guy in this context — will have to use what non-developers use, and they use what has a stronger marketing force as “this is an ISO standard”. This is why ISO standards (good or bad) should be in the focus of our attention.

Since the world already has an International Standards Organization, and since we learned it must be patched, it is our responsability — we, the developers, geeks, sysadmins etc — to be aware of and get involved with what ISO is standardizing right now and help the technical process of standardization to happen solely with technical arguments and not political interests.

Otherwise the future world will still be divided in two groups of developers: those that develop with/for good stuff and those that develop with/for stuff used by non-developers just because a powerful company had the strength to standardize it whatever it takes.

Upgrade to WordPress 2.5

This is my first post in the new WordPress 2.5.

Upgrade was as easy as:

$ cd avi.alkalay.net
$ svn sw http://svn.automattic.com/wordpress/tags/2.5/

Everything worked without any changes, including my experimental content-optimized Plasma theme.

The new WordPress has lots of improvements, specially in the administration part and I recommend it.

OOXML: Open Letter to YES-voting and ABSTAIN-voting Countries

I am learning a lot from all this standardization process.

I was a member of the brazilian committee and I also analyzed the specification. My country disapproved OOXML and I think this was a decision based on logic lead by the process.

The JTC1 rules page 49 item 9.8 says:

NBs may reply in one of the following ways:

  • Approval of the technical content of the DIS as presented (editorial or other comments may be appended);
  • Disapproval of the DIS (or DAM) for technical reasons to be stated, with proposals for changes that would make the document acceptable (acceptance of these proposals shall be referred to the NB concerned for confirmation that the vote can be changed to approval);
  • Abstention (see 9.1.2).

[Note: Conditional approval should be submitted as a disapproval vote.]

In other words, from my understanding, if there is one or more technical problems, the NB must disapprove the DIS. Many countries found many technical problems in OOXML that are still unresolved even after the BRM.

I also understand that such an important matter as an International Standard for Office Documents can’t be defined by 10 or 30 opinions collected as votes in a committee. Thats why the JTC1 process above talks about technical content, not opinion or vote. What I learned from studying the OOXML specification is that it is not ready for acceptance since many countries have found and reached consensus that the spec has problems, even after the BRM. If the NB-leveraged technical team — formed by people that would vote YES and NO — has produced a list of submitted problems in the spec, this list is by itself the consensus that the spec is still problematic.

I would like to understand why an NB that has produced technical comments voted YES or ABSTAINED. I thought abstention is a position for countries that were not able to create a committee to technically discuss the specification for reasons such as logistics or lack of quorum.

I am learning about all this and I’d like to have more solid arguments to build an opinion about these NB’s maturity to run a well documented process.

OOXML: Venezuela changed vote from YES to NO

Check it out:

From: “***FONDONORMA***” <in*&%@fondonorma.org.ve>
To: <ki*%$#@itscj.ipsj.or.jp>, <br*%$#%@iso.org>, <tak*%$#%@iso.org>, <ga*%$#%@iso.org>, “Maria Teresa Saccucci” <mar*%$#%@fondonorma.org.ve>, “Norma Arias” <norm*%$#%@fondonorma.org.ve>, “***FONDONORMA***” <in*%$#%@fondonorma.org.ve>
Date: Fri, 28 Mar 2008 14:04:26 -0430
Subject: Modification to the vote on DIS 29500 – Venezuela (FONDONORMA)
Toshiko Kimura
Secretary ISO/IEC JTC 1/SC 34

Att.: Mr. Keith Brannon, Mr. Maho Takahashi, Ms. Martine Gaillen

Dear Mr. Kimura,

Attached please find a letter from Mrs. María Teresa Saccucci, Standardization Manager of FONDONORMA (Venezuela) through which Venezuela wishes to modify its position on DIS 29500, Information technology – Office Open XML file formats from “Approval with comments” to “Disapproval with comments”.

Best regards,

Leonardo Di Bartolo
Coordinator of International Relations
FONDONORMA
Venezuela

This NO was a result of a very difficult meeting. Consensus was not reached and a lot of confusion happened. Exactly as in the first brazilian NO.

The NB had to decide the final vote based on technical issues still open in OOXML. The problem was the method of deciding and those technical points were not presented. So people only had in their head the Microsoft arguments that I already knew were part half trues and part complete lies.

I participated in the OOXML process in 3 countries and I was able to build an opinion on how normalization happens today in the world, and I’ll show some ideas in a future post.

By the way, this is how Caracas looks like, a city surrounded by huge green mountains that have behind them the Caribbean Sea.

Caracas from the EuroBuilding hotel

Caracas and the Caribbean Ocean in the map
Caracas from the EuroBuilding hotel

OOXML’s BRM: A flow of dirt

Finally Jomar, one of the brazilian delegate that went to OOXML’s BRM in Geneva has started to tell all the dirty little details of what really happened in that meeting and the surreal modus operandi of how 120 people can discuss 1027 issues in 5 days. Have fun in english and portuguese.

Oh, and talking about dirty playing, check the domain www.DocumentFreedomDay.com but remember that the original one is www.DocumentFreedom.org. The first one really deserves a DDoS attack.

OOXML: Brazil Says NO. Again.

It is now official. Brazilian vote was decided by consensus of the entire technical team, including Microsoft crew’s: OOXML does not deserve to be an international ISO standard.

Our first vote, in august, was also NO, due to the same reasons: OOXML is an awful specification.

That outcome was expected because we simply followed the process: technically analyze the OOXML specification, make comments, wait for responses, analyze them and see if all problems were fixed. Is there any single remaining unresolved problem? Vote NO. And in fact there were many many unresolved problems.

If every country followed this simple process, OOXML would receive a NO from 100% of them.

But in some countries, how is the process? Invite a few companies and simply count their votes. The problem here: 10, 20 or 80 votes can never represent what is the best for that country. Only, maybe, if you collect one vote for each citizen.

What I am trying to say is that in this case a decision must be reached by technical consensus, not vote. It is not a matter of will, but a technical issue that can only be reached by rational analysis and deliberation.

In Chile for example, 21 voting companies will define a 15 million people country vote in ISO. How easy is to lobby these 21 companies with Power Point presentations telling complete plain lies ?

Technically speaking, if your country’s vote was YES or ABSTENTION, one of these possibilities happened:

  1. Nobody had time to analyze the OOXML specification and the ABSTENTION was the right choice.
  2. Nobody had time to analyze the OOXML specification and a few people decided for you to vote YES, based on ideology or a result of lobby, not technology benefits.
  3. Even having time to analyze the OOXML specification, a few people decided for you to vote YES, based on ideology or result of lobby, not technology benefits.

Seems stupid, but these are exactly the 2 possibilities of OOXML getting YES votes in ISO. It is still an awful specification.

By the way, Brazil would vote NO again and again and again even if all OOXML’s technical issues could be resolved. OOXML would still have legal issues and also serious overlap problems with the OpenDocument Format ISO standard.

I was not present in today’s meeting in ABNT because I already knew what would be the result, since the process of analysis and deliberation in Brazil was very strictly followed. Hopefully Jomar will write about it and you can check more details.

Open Source Software inside the iPhone

Can Open Source Software be more ubiquitous than this ?

A few days ago I was playing extensively with Apple’s iPhone, investigating each sub-menu and little details. There is a section listing legal stuff and software being used with each license. GPL, LGPL, BSD and other Open Source licensed software rule the iPhone.

Some I have noted:

Microsoft sells Intra-operability as Interoperability

Interoperability

This is real Interoperability

  • Technical specifications for global interoperability and freedom
  • Interoperate through standards
  • Software product is less important
  • Switch products and keep access

Microsoft started their so called “interoperability” initiative. With a deeper look we find that the main objective of the released specifications is to let developers interact with their products only, something I like to call Intra-operability.

Intraoperability

This is Intraoperability

  • Technical specifications for interaction with and favor the product
  • Interoperate through the product
  • Software product is the main player
  • Switch product and loose access

See the difference?

Microsoft technical specifications have serious technical and legal issues, and are being released not to increase interoperability across people, organizations and applications, but to leverage the use of their products. For the standard document format scenario: OOXML is about allowing other people to interact with files that are primarily generated and manipulated with Microsoft Office. It is not about full interoperability, which would enable competition with Microsoft Office, for obvious reason.

For each purpose or scope, better open standards exist and should be adopted and used instead: Java versus .NET, XHTML versus IE-campatible-only DHTML, ODF versus OOXML, etc

My friend Cezar Taurion has also written some words about Intra-operability (in portuguese). Bob Sutor also put together some words about it.

These graphics are available in a friendly Creative Commons license, in this animated ODF file, in case you want to integrate them in your presentations. PDF export also available.

How Microsoft raped ISO

One of the most critical and discussed points of the whole OOXML subject is how the specification lets you include binary proprietary information.

Let me show you how it happens with a piece of an OOXML document, the red-marked text is the problematic part (see for yourself, §6.2.2.14, paper page 4,813, lines 7–13):

<v:shape>
<o:ink i="AMgFHQSWC+YFASAAaAwAAAAAAMA..." annotation="t" contentType="application/x-ms-ink"/>
</v:shape>

So you, as a programmer, please tell me what to do if you are developing an application that must read and generate this kind of document. How can I find documentation about this encoded binary stream? Is this a good practice in XML? Anyway, this kind of (bad) design appears in many parts of the OOXML spec. Want more examples of bad design? Have fun.

Suppose I really want to develop this kind of support in my application, I am a master programmer and reverse-engineered a few examples generated with a copy of MS Office 2007 that I had to buy. Or maybe I just found the specification of this proprietary application/x-ms-ink type and go to develop a library to handle it.

Inevitably, my library will reimplement aspects of some Microsoft library with same functionality, and according to a Software Freedom Law Center report, the Microsoft Open Specification Promise (OSP, basically the not-open-enough license Microsoft lawyers wrote for the usage and implementation of OOXML) cover’s the specification only, not code.

I will be sued for patent infringement. On software rewrite, not on specification usage.

Then the rape begins. Microsoft is claiming that this was solved in the ISO’s BRM meeting. This is the solution, from the ridiculous 12 pages resolutions, page 7:

Resolution 25: The BRM decides to accept the editing instructions contained in http://www.itscj.ipsj.or.jp/sc34/def/BRM/Response_0135_bitfields.doc in place of R 135, replacing “deprecated” by “transitional”, and with the following addition: The Editor shall ensure that all existing attributes defining the bitfields described above shall be “transitional”—so resolved.

Who reads this resolution with high level eyes may think that all binary fields will be removed from the specification. But please, you, as a programmer, tell me how the so called Editor will find all and every single part of a 6000+ pages specification containing bitfields? How he’ll expand all those bitfields in an XML subspecification? Will he invent some? Is he the right person (or team) to do that?

When done and if done, the specification will be something completely new, full of new parts. Will jump from 6063 pages to maybe 7500. Oh, and did I mention it will be something that even MS Office 2007 or 2008 don’t support today? Supporting or not, implemented or not, this new unexistent specification is the “thing” that countries’ National Bodies are voting right now, without even seeing it, without checking if it was corrected, without having time for this because they didn’t receive it for revision.

They won’t have time to review 6000+ pages because the deadlines defined by the ISO’s Fast Track process are over.

So the question is how ISO/IEC and JTC1 let such a big and problematic specification enter the light Fast Track process? My answare: ISO was raped.

I’ve been talking to several people that will define their country’s vote and their mindset is “are you really putting ISO in such a bad position?” Well, yes. You know, in the end ISO is not god. They are a bunch of people that, like you and me, have religions, aspirations, problems, family, go to the bathroom etc. Like you and me, they may be also naive in regard to some subjects, particularly document formats. Then comes Microsoft excellent speakers showing PowerPoint charts that are plain lies (e.g. “OpenOffice.org supports OOXML”) and people believes them.

People have two choices: to question ISO’s reputation in the OOXML case, or question IBM, Sun, Oracle, Red Hat, Free Software Law Center, ODF Alliance and many other institutions’ reputation when they massively scream in chorus that OOXML has serious technical, legal and standardization process issues.

This is a world where organizations like ECMA has completely lost the respect of the technical community. But this is not a big problem, because we have other similar, still reliable bodies as OASIS, W3C, OSF, etc.

This is a world where we will have to work hard to make ISO regain its (currently damaged) reliable status. This is a big problem, because we only have one (and only need one) International Standards Organization.

This is your responsibility. To start this work get involved with your country’s National Body for standardization and promote the creation of a formal letter to ISO about the OOXML process, its problems and how ISO let that happen. INCITS in USA, ABNT in Brazil, INN in Chile etc.

Better sleep leads to a better You

The title was the phrase written in a CD left in a Crown Plaza Hotel room I stood this days.

Its excellent content is a relaxation program with very soft music to listen and do after you go to bed and before you fall asleep, with the aim to make you sleep better.

I found it so effective that I read more through the CD cover. It is produced by a company called SoundSleep with the help of a certain Dr. Michael Breus.

The cover also contained some other “holistic approach to sleep principles” which I also find useful and would like to share:

  • Relax before bedtime. Stress can make you miserable and restless. Take some time for a pre-sleep ritual to break the connection between stress and bedtime. Try listening to the enclosed Sleep CD, reading, meditating, light stretching, lavender aromatherapy or a hot shower.
  • Watch the caffeine. Coffee and many teas and sodas contain caffeine and may keep you up. If you’ve already had too much, consider eating some carbohydrates like bread or crackers to help reduce the effects.
  • Watch the alcohol. Alcohol may initially help you fall asleep, but as your body clears it from your system, it can also cause nightmares, sweats and headache. Drink one glass of water for every alcoholic beverage consumed to try to reduce these symptoms.
  • Exercise at the right time. Regular exercise relieves stress and encourages good sleep. However, if a little exercise really gets your blood pumping, you’d be wise to not work out in the evening or just before bedtime.
  • Cut down on noise, light and extreme temperatures. Try earplugs, a night light, an eye mask or drape clip. The best temperature for sleep is 20 to 22°C (68 to 72°F).
  • East right, and sleep tight. Avoid eating a large meal just before bedtime or going to bed hungry. It’s about balance. Some foods that promote sleep include: milk, pumpkin, artichokes, avocados, almonds, eggs, peaches, walnuts, apricots, oats, aspargus, potatoes and bananas.
  • Understand jet lag. Before you cross time zones, try waking up later or earlier to help the body adjust to time difference. And remember, it takes a few days for the body to catch up.
  • Remember the purpose of the bed. Avoid TV, eating, and emotional discussions in bed. The mind and body associate bedtime activities with being in bed. So don’t let a bad habit keep you awake.
  • No drinks after 8 p.m. It’s a fact; most of us cannot simultaneously go to the bath room and sleep. So shut down your fluid intake before 8 p.m. so you can get your rest.
  • Nap smart. A little 20-minute power nap during the early part of the day can really be refreshing. But sleep too much, and you may spend the night staring at the ceiling.

How I Became Vegetarian

Around early 1995 I was thinking about to stop eating meat. I thought that would do good for my body and my mind.

Then I was waiting in a room, randomly took a book and randomly open it in the part that finally defined my decision. Later a saw many people becoming vegetarian because of the same part from the same book, so I found it important to share. The author was Sri Yukteswar and the book name is The Holy Science, from 1894.

This is the part of the book that influenced my decision that time (copy and pasted from Sain Louis’ Guide to Vegetarian Life):

What is natural food for man?

First, to select our natural food, our observation should be directed to the formation of the organs that aid in digestion and nutrition, the teeth and digestive canal; to the natural tendency of the organs of sense which guide animals to their food; and to the nourishment of the young.

Observation of teeth

By observation of the teeth we find that in carnivourous animals the incisors are little developed, but the canines are of striking length, smooth and pointed, to seize the prey. The molars also are pointed; these points, however, do not meet, but fit closely side by side to separate the muscular fibers.

In the herbivorous animals the incisors are strikingly developed, the canines are stunted (though occasionally developed into weapons, as in elephants), the molars are broad-topped and furnished with enamel on the sides only.

In the frugivorous all the teeth are of nearly the same height; the canines are little projected, conical, and blunt (obviously not inteded for seizing prey but for exertion of strenth), The molars are broad-topped and furnished at the top with enamel folds to prevent waste caused by their side motion, but not pointed for chewing flesh.

In Omnivorous animals such as bears, on the other hand, the incisors resemble those of the herbivorous, the canines are like those of the carnivorous, and the molars are both pointed and broad-topped to serve a twofold purpose.

Now if we observe the formation of the teeth in man we find that they do not resemble those of the carnivores, neither do they resemble the teeth of the herbivorous or th eomnivorous. They do resemble, eactly, those of the frugivorous animals. The reasonable inference, therefore, is that man is a frugivorous or fruit-eating animal.

Observation of the digestive canal

By observation of the digestive canal we find that the bowels of carnivorous animals are 3 to 5 times the length of their body, measuring from the mouth to the anus; and their stomach is almost spherical. The bowels of the herbivores are 20 to 28 times the length of their body and their stomach is more extended and of compound build. But the bowels of the frugivorous animals are 10 to 12 times the length of their body; their stomach is somewhat broader than that of the carnivorous and has a continuation in the duodenum serving the purpose of a second stomach.

This is exactly the formation we find in human beings, though anatomy says that the human bowels are 3 to 5 times the length of man’s body – making a mistake by measuring the body from the crown to the soles, instead of from mouth to anus. Thus we can again draw the inference that man is, in all probability, a frugivorous animal.

Observation of organs of sense

By observation of the natural tendency of the organs of sense – the guideposts for determining what is nutritious – by which all animals are directed to their food, we find that when the carnivorous animal finds prey, he becomes so much delighted that his eyes begin to sparkle; he boldly seizes the prey and greedily laps the jetting blood. On the contrary, the herbivorous animal refuses even his natural food, leaving it untouched, if it is sprinkled with a little blood. His senses of smell and sight lead him to select grasses and other herbs for his food, which he tastes with delight. Similarly with the frugivorous animals, we find that their senses always direct them to fruits of the trees and field.

In men of all races we find that their senses of smell, sound, and sight never lead them to slaughter animals; on the contrary they cannot bear even the sight of such killings. Slaughterhouses are always recommended to be removed far from the towns; men often pass strict ordinances forbidding the uncovered transportation of flesh meats. Can flesh then be considered the natural food of man, when both his eyes and his nose are so much against it, unless deceived by flavors of spices, salt, and sugar? On the other hand, how delightful do we find the fragrance of fruits, the very sight of which often makes the mouth water! It may also be noticed that various grains and roots possess an agreeable odor and taste, though faint, even when unprepared. Thus again, we are led to infer from these observations that man was intended to be a frugivorous animal.

Observation of the nourishment of the young

By observation of the nourishment of the young we find that milk is undoubtedly the food of the newborn babe. Abundant milk is not supplied in the breasts of the mother if she does not take fruits, grains, and vegetables as her natural food.

Cause of disease. Hence from these observations the only conclusion that can reasonably be drawn is that various grains, fruits, roots, and – for beverage – milk, and pure water openly exposed to air and sun are decidedly the best natural food for man. These, being congenial to the system when taken according to the power of the digestive organs, well chewed and mixed with saliva, are always easily assimilated.

Other foods are unnatural to man and being uncongenial to the system are necessarily foreign to it; when they enter the stomach, they are not properly assimilated. Mixed with the blood, they accumulate in the excretory and other organs not properly adapted to them. When they cannot find their way out, they subside in tissue crevices by the law of gravitation; and being fermented, produce diseases, mental and physical, and ultimately lead to premature death.

By the way, I am not 100% vegetarian today. I still have seafood very rarely, when is very difficult to find veggy options in some fancy restaurants.

OOXML in Numbers

This is a numerical photography of what is the Microsoft Office Open XML wannabe specification:

  • 6063 pages written in 12 months
  • Only 6 months of analysis pointed 3500 issues that span from intellectual property, to standard date & time representation, to malformed XML definitions, to binary proprietary bits
  • 3500 issues where summarized in “only” 1027
  • ECMA proposed 2293 pages of mutually conflicting changes to make happy those 3500 comments
  • Only 30 days to analyze those 6063+2293 pages against those 3500 comments
  • Only 5 days of BRM to discuss 6063+2293 pages
  • Only 18% of these 1027 problems where discussed in BRM
  • 82% of problems were not even discussed
  • BRM produced a short 12 pages document containing alterations to those 6063 pages long spec based on 2293+thousands of pages of comments from all countries+19 months of technical writing and global analysis work. By the way, those short 12 pages propose such nebulous alterations — as Resolution 25 on page 7 — that it is difficult to foretell how they will technically fit in the new specification. If applied, those alterations could turn the original spec upside down, so deeply that it could become a completely new specification, maybe even worse than the first one.
  • Now countries must vote if they approve or not this still unfinished new OOXML that will be divided in 4 independent parts for strict conformance class+4 parts for transitional (deprecated) conformance class. These final multipart wannabe standards don’t exist yet and will be released about the same day countries will have to vote for it.

Or graphically:

History and confusion of OOXML

And you still think OOXML is ready to become an International Standard Format for storing YOUR documents ?

Come on, give me a break !

How to Make High Quality DVD Rips

Some friends asked so the following is how I encode (rip) DVDs.

Choosing the file format: .AVI, .OGG, .MP4 or .MKV ?

The ripped video file format is a decision you must make. Currently my format of choice is .MKV or Matroska. I’ll explain why.

It is quite idiotic to say that an .MP4 movie has better quality than a .AVI or vice-verse (or any other combination of comparisons). OGG, MP4 (MPEG-4 Part 14), MKV (Matroska), AVI, WMV (or ASF) are just containers, envelopes. Video quality depends on what goes inside it.

“Multimedia” has this name because you have multiple types of media: video in multiple angles, multiple audio options including different languages and channels (stereo, 5.1, 6 channels etc), subtitles in several languages, chapter information, menu etc. Think about a DVD. So this is a graphical view of how things are organized inside a 900MB movie file in a modern format as MKV or MP4:

Header with tags, track names, chapters info, seek positions Main Video track (MPEG-4 AVC/H.264) Attachments as JPG images, documents, scripts or text files
Video segment showing another angle (MPEG-4 ASP/Xvid/DivX)
Audio track: English Dolby Surround 5.1 (AC3)
Audio track: Director’s comments stereo (MP3)
Audio track: Portuguese Dolby Surround 5.1 (DTS)
Subtitle track: Portuguese (Unicode text)
Subtitle track: Chinese (Unicode text)
Subtitle track: English (VobSub)
byte 100K byte 100M byte 200M byte 310M byte 420M byte 530M byte 650M byte 780M byte 895M byte 900M

A digital multimedia file format must be capable to contain all this different medias and multiplex them in parallel so you won’t have the video in the first 500MB of the file and the audio on the following 500MB (this can’t work for streaming). And this is exactly what modern file formats as MP4 and MKV do: they carry all your movie-related data together.

This is a comparison of all these file formats based on my personal experience with them (a more formal comparison can be found in Wikipedia):

.MKV .MP4 .AVI
Industry support Almost none Good and increasing, specially on Apple platforms, the mobile scene and Nero Digital ecosystem Treated as legacy popular format
Usage on the web Very popular on HD or high quality DVD rips Very popular on HD or high quality DVD rips, supported by Flash Player, YouTube, Google Video Popular amongst low-quality DVD rips
Support for advanced video formats and multiple video angles Yes. MPEG-4 ASP (DivX, Xvid), MPEG-4 AVC (a.k.a. H.264) etc Yes. Only MPEG-4 systems and a few others Problematic and No
Support for multiple audio tracks (channels, formats, languages and “director’s comments”) Yes Yes. Formats are only MP3, AAC and a few others not very popular Yes
Support for tags (artist, title, composer, etc as MP3’s ID3) Yes Can be supported by MP4 extensibility but this is not very well standardized across authoring tools (iTunes, GPAC etc) and players (Amarok, Media Player Classic, iPod, Windows Media Player etc) No
Support for attachments with mime-types (used to attach movie posters images or other files) Yes No
Support for chapter marks Yes No
Support for multiple language embedded soft-subtitles Yes. VobSub (as extracted from DVDs), plain timed UTF-8 text (SRT, SUB) etc No
Support for naming tracks with human names as “Director’s comments” or “Portuguese subtitles” etc Yes Yes No
Support for carrying menus information (as in DVDs) and interaction Yes through an XML idiom, but unsupported by most players Yes through SVG, but unsupported by most players No
The container overhead in bytes in the final file Very small Very small Very big
Supported by free and Open Source multiplatform authoring tools Perfect on Linux, Unix, Windows and Mac Yes (see GPAC), with some intellectual property issues. Tools need to mature. Yes
Ready for popular web streaming as in Flash player No Yes. The popular Flash Player that is installed on everybody’s browser supports playing MP4 files as long as they contain H.264 video and AAC audio tracks. I recommend the LongTail FLV/MP4 Player since it also plays subtitles embedded in MP4 files. No

Personally I believe MP4 is the multimedia file format for the future because since it is getting popular, all these non–standardized features will get stabilized. MP4 is an ISO standard and the increasing industry support can be felt on iPods and portable devices, and most notable on home DVD players capable of playing the 700MB MP4 video file burned in a CD.

By the way, remember this:

  • MP4 is not an evolution of MP3. AAC (MPEG-4 Part 3) is.
  • MP5 and MP6 (used to classify portable media players) are things that simply doesn’t exist in the multimedia scene.
  • .M4A, .M4V, .MOV and .3GP files can safely be renamed to .MP4. They just use to be different extensions to help user’s eyes easily identify whats inside.

Meanwhile, MKV wins everything but on the Industry Support category. But this doesn’t really matter, and I’ll explain why. Since MKV is just a container, the large video, audio etc streams can be extracted and repackaged into MP4 and vice-versa in seconds. No transcoding (decoding followed by a lossy encoding into another format) is needed.

So today I store my videos in the most feature rich and well supported by players format: MKV.

OGG or OGM (the container file format) is practically dead in my opinion. They were created as part of the Xiph initiative for a complete open source patent-free multimedia framework, but seems nobody uses it anymore for video. From the same family, Vorbis (the audio codec compared to MP3, a.k.a. .OGG) is very good but also very not popular. Theora (the video codec) is frequently comparable to old MPEG-1 in terms of quality and compression ratio so currently, if you want quality and are not concerned about patents, MPEG-4 AVC is the best choice. FLAC, Xiph’s lossless audio codec, is the winner of the family: very popular, massively used, and recommended.

Encoding the DVD

I use HandBrake, the most practical Open Source (and overall) movie encoder. It runs on Linux, Mac and Windows and uses the same Open Source libraries as ffmpeg, mplayer/mencoder, xine, etc. While these programs are generic video handlers (with thousands of confusing configuration parameters to sustain this generalistic status) HandBrake is optimized only for ripping so it is very easy to use, yet extremely powerful.

#!/bin/bash

##
## This is the script I use to make hifi DVD rips including chapter markers and
## subtitles. It uses Handbrake.
## Contains what I found to be the best quality ripping parameters and
## also let me set simple parameters I need.
##
## Avi Alkalay <avi at unix dot sh>
## http://avi.alkalay.net/2008/03/mpeg4-dvd-rip.html
##
## $Id$
##

#set -vx

HANDBRAKE=${HANDBRAKE:=~/bin/HandBrakeCLI}
#HANDBRAKE=${HANDBRAKE:="/cygdrive/c/Program Files/Handbrake/HandBrakeCLI.exe"}
## Where is the Handrake encoder executable.
## Handbrake is the most practical free, OSS, DVD riper available.
## Download HandBrake for Linux, Mac or Windows at http://HandBrake.fr


INPUT=${INPUT:=/dev/dvd}
## What to process. Can also be a mounted DVD image or simply '/dev/dvd'


TITLE=${TITLE:=L}
## The title number to rip, or empty or "L" to get the longest title


#CHAPTERS=${CHAPTERS:=7}
## Example: 0 or undefined (all chapters), 7 (only chapter 7), 3-6 (chapters 3 to 6)


#VERBOSE=${VERBOSE:="yes"}
## Wether to be verbose while processing.


SIZE=${SIZE:=1200}
## Target file size in MB. The biggest the file size, the best the quality.
## I use to use from 1000MB to 1400MB for astonishing high quality H.264 rips.


OUTPUT=${OUTPUT:="/tmp/output.mkv"}
## Output file. This will also define the file format.
## MKV (Matroska) is currently the best but MP4 is also good.


AUDIO=${AUDIO:="-E ac3 -6 dpl2 -D 1"} # For AC3 passthru (copy).
#AUDIO=${AUDIO:="-E lame -B 160"} # For MP3 reencoding. Good when input is DTS.
## Audio parameters. If input is AC3, use it without transcoding.
## If is DTS, reencode to MP3.


MATRIX=${MATRIX:=`dirname $0`/eqm_avc_hr.cfg}
## x264 matrix to use. The matrix file may increase encoding speed and quality.
## This one is Sharktooth's as found
## at http://forum.doom9.org/showthread.php?t=96298

######### Do not change anything below this line ##############


## Make some calculations regarding title and chapters based on parameters.
SEGMENT=""
if [[ "$TITLE" == "L" || -z "$TITLE" ]]; then
	SEGMENT="-L"
else
	SEGMENT="-t $TITLE"
fi

[[ -n "$CHAPTERS" && "$CHAPTERS" -ne 0 ]] && SEGMENT+=" -c $CHAPTERS"

[[ "$VERBOSE" != "no" ]] && VERB="-v"



# Define args for the x264 encoder. These are some values I found on the net
# which give excelent results.
X264ARGS="ref=3:mixed-refs:bframes=6:b-pyramid=1:bime=1:b-rdo=1:weightb=1"
X264ARGS+=":analyse=all:8x8dct=1:subme=6:me=umh:merange=24:filter=-2,-2"
X264ARGS+=":ref=6:mixed-refs=1:trellis=1:no-fast-pskip=1"
X264ARGS+=":no-dct-decimate=1:direct=auto"
[[ -n "$MATRIX" ]] && X264ARGS+=":cqm=$MATRIX"


# Encode...
"$HANDBRAKE" $VERB -i "$INPUT" -o "$OUTPUT" \
	-S $SIZE \
	-m $SEGMENT \
	$AUDIO \
	-e x264 -2 -T -p \
	-x $X264ARGS


# Repackage to optimize file size, to include seek and to include this
# this script as a way to document the rip...
echo $OUTPUT | grep -qi ".mkv"
if [[ $? && -x `which mkvmerge` && -f $OUTPUT ]]; then
	mv $OUTPUT $OUTPUT.mkv
	mkvmerge -o $OUTPUT $OUTPUT.mkv \
		--attachment-name "The ripping script" \
		--attachment-description "How this movie was created from original DVD" \
		--attachment-mime-type application/x-sh \
		--attach-file $0

	[[ -f $OUTPUT ]] && rm $OUTPUT.mkv
fi

The script seems long because it is fully documented but it actually only collects some parameters and simply runs the HandBrake encoder like this (passed parameters are in red):

~/bin/HandBrakeCLI -v -i /dev/dvd -o /tmp/output.mkv \
    -S 1200 \
    -m -L \
    -E lame -B 160 \
    -e x264 -2 -T -p \
    -x ref=3:mixed-refs:bframes=6:b-pyramid=1:bime=1:b-rdo=1:weightb=1:analyse=all:8x8dct=1:subme=6:me=umh:merange=24:filter=-2,-2:ref=6:mixed-refs=1:trellis=1:no-fast-pskip=1:no-dct-decimate=1:direct=auto:cqm=~/src/randomscripts/videotools/eqm_avc_hr.cfg

All the rest is what I found to be the best encoding parameters.

The resulting video file (/tmp/output.mkv in this case) will contain correctly cropped video and audio quality as good as the DVD (it is almost lossless), and chapter breaks at the same positions read from the DVD.

In a Core Duo machine as my laptop running Fedora 8 or Windows XP, a 2 pass H.264 encoding (2 pass improves quality and H.264 is newer standard MPEG-4 technology better than DivX/Xvid) takes about 4 to 5 hours for a regular 2 hours movie, so leave it encoding while you go to sleep. A Pentium 4 machine running Ubuntu takes about 17 hours for the same rip.

I use to rip one chapter from the movie first (use your preferred video player or lsdvd command to find the shortest chapter), check quality, compare to DVD, fine tune, try again and then shoot full DVD ripping.

After encoding I use to repackage the audio/video stream with Matroska‘s mkvmerge (or mmg, its GUI version available on any Linux distribution as “mkvtoolnix” package, and installable for Windows or Mac OS from Matroska’s website) to optimize seeks and to include soft subtitles (that can be turned on and off as on regular DVDs), but I’ll explain that in another HOWTO.

Give Your Ripped Movie a Descriptive File Name

I use to organize my media library in a standard way I invented for myself and which I suggest you to use too.

My movie file names shows everything that the file includes. Some examples:

  • Indiana_Jones_and_The_Raiders_of_the_Lost_Ark_IMDB{tt0082971}-Xvid{720x304_23.98fps}+MP3{ENG,POB_VBR}+Sub{ENG,SPA,POB}+Covers.mkv
    This is the Indiana Jone’s Raiders of the Lost Ark movie, whose IMDB index is tt0082971 (IMDB{tt0082971}). It was ripped with the old Xvid codec and contains 720×304 pixels frame size at a rate of 23.98 frames per second (Xvid{720x304_23.98fps}). It also contains selectable audio tracks in English and Brazilian Portuguese encoded in variable bit rate MP3 (MP3{ENG,POB_VBR}). In addition, there is also selectable subtitles in English, Spanish and Brazilian Portuguese (Sub{ENG,SPA,POB}). The file also contains the cover images as attachments.
  • Harold_and_Maude_IMDB{tt0067185}-H264{672x368_3Pass_25fps}+HEAAC{EN}+Sub{POR,EN,FRE}+Chapters+Covers.mkv
    The old Harold and Maude movie whose IMDB index is tt0067185 (IMDB{tt0067185}). It is encoded with H.264 in 3 passes and has 672×368 pixels frame size at a rate of 25 frames per second (H264{672x368_3Pass_25fps}). There is only one English audio track encoded in modern HE-AAC (HEAAC{EN}). Subtitles in Portuguese, English and French (Sub{POR,EN,FRE}), chapter information and attached cover images. This is very complete high quality DVD backup.
  • I_Am_Legend_IMDB{tt0480249}-H264{704x304_23.98fps}+AC3{ENG_5.1}+Sub{POR}.mkv
    The I Am Legend movie whose IMDB index is tt0480249 (IMDB{tt0480249}), video encoded in H.264 with 704×304 pixels frame size (H264{704x304_23.98fps}), original 5.1 channels AC3 audio in English (AC3{ENG_5.1}) and subtitles in Portuguese (Sub{POR}).

The advantages of this scheme are:

  • It is web safe with no spaces in filenames. All underlines. It is also DOS safe.
  • To have the IMDB index let me know exactly which movie this file contains. This is particularly good to avoid ambiguity for movies that have remakes as Ben Hur, or movies that have an official name but are well known by other names or have international titles.
  • To know the encoding method, subtitles included and chapters info give me the overall quality of the movie right away.
  • Special attention to audio and subtitle languages. Having them on the filename let me know I will understand its content without having to play. Sometimes I can’t play the file because I logged in my home computer remotely.

Playing the Ripped File

To play this advanced Matroska media file that contains such a rich set of metainformation and highly compressed digital content you will need an advanced player too. And happens that the best players are the Open Source ones. So use these players:

These are Media Player Classic screenshots demonstrating how to activate the advanced content inside a Matroska file. Players on other platforms have similar capabilities and menus.

Activating embedded subtitles and languages
The player lets you choose the audio language and subtitles. On MPC for example, this is how you turn on and off and choose the language for subtitles.
Choosing subtitles language

As you can see, the player found subtitles embedded in the MKV file in English, Hebrew and Portuguese.

If the MKV file contains many audio tracks (as different languages, director’s comments etc) this is how to select it:

Selecting audio track to play in Media Player Classic

And to jump directly to a specific chapter on the movie, if the MKV file contains this kind of information:

Using Media Player Classic to browse chapter in a movie file

Improving audio volume
If you ripped the movie without reencoding the audio, the final file will contain DVD’s original AC3 audio tracks in 6 channels (5+1). This may sound with a lower volume when played in a 2-speaker system as your laptop, iPod, etc because 4 channels are simply not being played. To remediate this the player will have to downsample the audio. In other words, it will remix the 6 channels into 2 stereo channels while playing. The Media Player Classic options should look like this:

Selecting Media Player Classic\'s Audio Options

Configuring audio downsample on Media Player Classic

ODF in the State of Paraná

The state of Paraná, here in Brazil, is the first to have laws that specifically promotes the use of OpenDocument Format. This is a free translation of such law:

  1. All public companies and institutions under the state of Paraná administration will adopt, preferably, open formats for creation, storage and public availability of digital documents.
  2. We understand open formats as the ones that:
    1. promote interoperability between multiple applications and platforms, internal and externally;
    2. let users chose applications free from royalties;
    3. can be freely and independently implemented by multiple computer software vendors, in multiple platforms, without any charge for the intellectual property of the technology.
  3. The companies cited in point 1 must be enabled for receiving, publishing, visualization and preservation of digital documents in open formats according to norm ISO/IEC 26300:2600 (OpenDocument format – ODF).

This law was signed by Roberto Requião, the state gorvernor, and Nizan Pereira Almeida, secretary for strategic initiatives.

Other states in Brazil, inspired by Paraná, are creating similar laws.

Folks related to the ODF initiative in Paraná

The folks in the photo, left to right, are: Henrique Menezes, government relations, IBM. Jomar Silva, ODF Alliance chapter Brazil. State congressman Edson Praczyk. Vitorio Furusho, Celepar (a state public company), BrOffice.org users group and Open Source Movement in Paraná. Reginaldo Radel, congressman Prazyk assistent.

Web 2.0 Party

Web 2.0 is about people. Its a global party where human beings exchange knowledge, experiences, information and even emotions.

Yesterday I saw the Xanadu movie again with ELO’s All Over the World, a good-vibe song from the 80’s that still has very current lyrics that explain what people are doing in our 2008’s Web 2.0.

Check out the movie excerpt with the song and the lyrics.

Everybody all around the world
Gotta tell you what I just heard
There’s gonna be a party all over the world

I got a message on the radio
But where it came from I don’t really know
And I heard these voices calling all over the world

All over the world,
Everybody got the word
Everybody everywhere is gonna feel tonight

Everybody walkin’ down the street
Everybody movin’ to the beat
They’re gonna get hot down in the U.S.A.
(New York, Detroit, L.A.)

We’re gonna take a trip across the sea
Everybody come along with me
We’re gonna hit the night down in gay Pareee

All over the world,
Everybody got the word
Everybody everywhere is gonna feel tonight

London, Hamburg, Paris, Rome, Rio, Hong Kong, Tokyo
L.A., New York, Amsterdam, Monte Carlo, Shard End and

All over the world,
Everybody got the word
Everybody everywhere is gonna feel tonight

Everybody all around the world
Gotta tell you what I just heard
Everybody walkin’ down the street
I know a place where we all can meet
Everybody gonna have a good time
Everybody will shine till the daylight

All over the world,
Everybody got the word
Everybody everywhere is gonna feel tonight

All over the world
Everybody got the word
All over the world
Everybody got the word
All over the world
Everybody got the word

Software Security from a Specialist: Gary McGraw

I just listened to an excellent interview with Gary McGraw, a security guru, in the Software Engineering Radio, and I suggest software developers to hear it too.

Some points he touched:

  • Software security is about how to approach computer security if you are a developer or a software architect.
  • Security problems come from 2 points: (a) bad or buggy implementation as buffer overflows etc and (b) lack or poor architectural risk analysis. So even if you took a lot of care while writing the code you may have forgotten completely to authenticate users. This is a bad design (b) issue. Both problems — implementation and design — must be mitigated.
  • You can’t be 100% secure, but if you have considered security in the design and in the implementation of your software, you will be a lot better than simply shipping software without thinking about security.
  • Although people may have very good reasons to think that Open Source software is less secure than closed source because a cracker can see the code and find flaws, the bad guys actually use the binary version of a software to find the flaws, using low level debuggers, stack analyzers, decompilers and other kinds of things. Open Source software is not really in any worse shape that any other kind of software. He also says that Open Source software is not also better, from a security perspective. He does not believe in that theory that everybody is looking at the code and may find and fix bugs. Me neither.

I guess this is my last post of 2007 and I wish everybody a happy new year.

Anarchy at ISO

The fundamental question, lack of tangible answer and the probable consequences

This is a translation for a blog post by Jomar Silva, head of the ODF Alliance Chapter Brazil.

In the coming days I’ll celebrate (or regret) one year working with OpenXML in ISO, and I must admit that the more time goes by I’m more far of finding a plausible answer to the most fundamental (and forgotten) question: “Why two standards ?”

The claim of proponents is the legacy support, which is not technically proven in more than 6 thousand pages of specification. It is also not proven the claim that the OpenXML fulfills the specific users needs… Did any of them read the specification of ODF (ISO / IEC 26300)?

The commercial reasons for the existence of this second standard is more than obvious and have been widely commented worldwide, but doesn’t it work as a warning that something is going wrong? Does the initiatives of international standardization are just moved by commercial interests of six guys and the argument that standardization helps to reduce artificial barriers to free trade is nothing more than cheap and utopian ideology?

Would the ISO 9000 be what it is for Quality worldwide if its opponents had proposed and made whatever was needed to ensure many different standards of quality, addressing different user needs? Will ISO accept in the coming years a proposal for more flexible quality standard, which is compatible with the legacy of disorganization that some companies still have today? Does the lack of ability of a small company to have and maintain decently a quality system based on the ISO 9000 configures “specific users needs” and therefore demand a new international standard?

Let us move to environmental issues? Do small and medium businesses have conditions and structure to maintain an ISO 14000 certificate correctly? Do the allegations of China on emissions of carbon (by the way, supported by USA position) configures the existence of the specific user needs in order to prepare a new environmental standard?

The precedent that OpenXML brought to international standardization, in my point of view, is the worst possible because based on the existence of two standards for editable documents, or at least the ability to mobilize the world and spend huge amounts of money in this discussion, will not allow the JTC1, the ISO, IEC and anyone else to refuse the discussions of new rules or alternatives standards for any economic sector. This really worries me, because all the efforts of standardization achieved in recent decades can go up in the garbage can in a short time and the worst is that everyone seems not to see it is happening. Want to see an example of the mess… I give one of the bests…

The PDF format is an ISO standard, the PDF/A which is the ISO 19005-1:2005, published in October 2005. It was developed based on a subset of the specification PDF 1.4 of Adobe. Several countries and organizations around the world have already adopted this standard as the default for non-editable documents.

Microsoft has released Windows Vista along with a new standard for non-editable electronic documents, called XPS (XML Paper Specification), which uses the same packaging concepts of OpenXML (OpenPackaging Convention) to represent non-editable documents. Who guess where XPS is standardized as another “Open Standard” wins a gift… ECMA… A lot of creativity is not needed to understand that this standard will also be submitted to ISO through a FasTrack in the coming months, just following the OpenXML opened path to that.

Adobe has realized this and has already expressed itself. A press release from the company itself, from January of this year already announced the delivery of version 1.7 of the standard to ISO. This proposal has followed their natural path and this week the vote of the standard in ISO has finished. According to the expectation of a blog from Adobe, published yesterday, the standard was adopted. My personal opinion is that this approval is very important, because now PDF becomes a full international standard, not a subset as was the PDF/A.

So I warn all involved and concerned to allocate a part of their budget for 2008 to discuss the FasTrack of XPS, because I think it will be inevitable.

I wonder what will be the other standards that will be run over (or who tries to trample) within ISO. How much money will be spend in this decade to learn this lesson?

What makes me more disappointed is to see that all these things can put ISO in a delicate position in the international standardization scene. The mess ISO is letting to happen in its own rules may cause that to be an ISO standard or a John Doe’s standard will have same value (at least to IT industry). I think this whole mess threatens the reputation that this entity has, which for me has always been synonymous with seriousness and responsibility.

The existence of unique standards, built through community is what today allows me to access the internet and write this article and that allows you to read it here from anywhere, using any browser and any operating system. This is what allows us to buy any CD with music and to use in any CD player (from $50 thousand to $1.00). This is the world created by international standards but looks like its not good for everybody.

Commodity market competition is for serious companies, competent and committed with the differentiation to their customers… It is not for anyone, regardless of their size or achievements in the past…

To relieve a little bit the post, in the last weekend I decided to play my acoustic guitar and ended up playing a song that reminded me a lot about all that I wrote here. I think it encouraged me to write this article.

The song is “Anarchy in UK” by the Sex Pistols and when I played in the weekend, just switched the UK to ISO and things made sense… Perhaps this is the answer to the question that doesn’t have an answer…

The part that I found most interesting is:

Anarchy at the ISO
It’s coming sometime and maybe.
I give a wrong time, stop a traffic line.
Your future dream is a shopping scheme.

That is what they want… Anarchy at ISO!

Microsoft Does Not Own OOXML

This is a reply to David Nielsen’s “OOXML and Microsoft” post. Since comments are closed there, I am writing here.

David, there is nothing wrong in Microsoft opening such an important spec. We should welcome this step. But there are several things that we should take care here:

  1. There is already an approved, open, internationally standardized spec for office documents: ODF. So OOXML can be another spec, but can’t be another standard spec for the same purpose. The standardization push is not welcome. It is simply too late, amongst other issues.
  2. Microsoft opened their spec not because they wanted, but because they needed to, after ODF acceptance as the ISO standard. If Microsoft will be successful in its anti-ODF campaign, they won’t need this openness status any more and I bet OOXML v2 will be closed.
  3. Microsoft does not own the spec. OOXML is Ecma’s document. What MS Office implements is a something that is today compatible with Ecma’s OOXML. But Microsoft already announced they can’t wait for ECMA to add new releases. In fact, there is no guarantee that something like MS Office 14 file format will be open or integrated into ECMA’s OOXML spec. This is an excerpt from Microsoft’s Brian Jones blog and shows how Microsoft does not own OOXML:

    To your last point, it’s hard for Microsoft to commit to what comes out of Ecma in the coming years, because we don’t know what direction they will take the formats. We’ll of course stay active and propose changes based on where we want to go with Office 14. At the end of the day though, the other Ecma members could decide to take the spec in a completely different direction.

  4. Microsoft is trying to fool everybody saying that OOXML is different from ODF. ODF is “a standard for office documents” while OOXML claims to be “a standard for office documents and compatibility with legacy office file formats”, just to not have a 100% overlap. But the “compatibility” part makes no sense because OOXML is textual XML and the legacy formats are binary. OOXML and legacy .doc, .xls, .ppt may have similar organization or structure but is impossible to be “compatible”. But even if OOXML wants to have a relation with legacy, the spec is incomplete and does not provide a mapping like “this OOXML tag is related to that stream of binary bytes in the legacy format”. In the National Bodies meetings to discuss OOXML standardization we proposed to include this mapping (then legacy .doc would become a truly open spec), or remove the compatibility statement from the spec title. If removed, OOXML would still be a beautiful spec, but could not be accepted as an ISO standards because of its full overlapping with ODF.
  5. OOXML reinvents the wheel and has many obscure parts. As a specification, currently OOXML doesn’t have a quality grade to be a standard.

Creating OpenSearch plugins for Browsers

I just came across a Mozilla::Developer page that teaches how to let visitors on a site easily add that site’s search function into their browsers as a plugin.

If you are reading this in my blog and you select your browser’s search tool, this is what you’ll see:

OpenSearch option in browser’s tool

You’ll get the option to permanently add my blog’s search function to your browser. If you select it, you’ll have this:

OpenSearch option added to browser’s tool

To make it, I followed the instructions on the first link and created my OpenSearch description file. Look! Technorati, Microsoft, and many others have OpenSearch-enabled websites.

Zeroconf your Network

I remember the days when I was configuring DNS and DHCP for a small home network with only 2 or 3 computers.

This is not needed anymore since the invention of Zeroconf. As Wikipedia says, “is a set of techniques that automatically create a usable IP network without configuration or special servers. This allows inexpert users to connect computers, networked printers, and other items together and expect them to work automatically.”

Zeroconf got my attention when I installed Ubuntu Linux in one of my home PCs and it automatically started to show hostnames instead of IPs of my other home computers on the same DNS-less network. On my other Fedora Linux hosts, I had to manually install the avahi-tools and nss-mdns packages and I got the same functionality — as described in the Fedora Post-Installations Configurations.

Still without a local DNS server, each host can be pinged, SSHed, browsed, SMBed, etc using the hostname.local model, not the their IP anymore. So the machine with hostname floripa broadcasts itself as floripa.local. The same happens for all machines.

But I still missed this functionality when using my laptop booted on Windows. This OS was unable to understand the Zeroronf broadcasts until I installed the Apple implementation for Windows called Bonjour that can be downloaded from here.

To have a better, visual understanding of what Zeroconf can do for you, the Avahi website (Zeroconf implementation on Linux) provides a series of screenshots of regular applications discovering services in the LAN. Most notable is Konqueror — KDE’s file manager — using the zeroconf:/ URL to browse LAN services.

Now I finally know that my home doesn’t need things like Bind/DNS anymore.

Worshiping Joyce in Rio

Some time ago, my New Yorker cousin came to visit Brazil and I wanted to make sure she was leaving with her iPod full of brazilian specialities. A few weeks later we exchanged this conversation by e-mail.

  • “Im listening to music you sent me while writing (I really like Joyce!…although right now im listening to Paulo Bellinati)” — she said
  • Joyce is a godess. I worship her every time I need a good something-between-samba-and-bossa-nova in my ears.”— in my reply

Joyce’s voice singing Bossa-Nova is probably what gets closer to the true spirit of Rio de Janeiro city.

Ipanema beach, Rio de Janeiro

Enough talking. This is some of her works. While listening, close your eyes and imagine a calm sunday walk in Ipanema beach.

(pictures link to the location they where found)

New Laptop

I am sort of away this days because I got a new laptop. Check it out.

Lenovo T61

My previous laptop was a IBM Thinkpad T40, Pentium M, 512MB RAM, 1024×768 screen size, CD-RW, Cisco WiFi/b, no bluetooth. Served me well for 4 years.

The new one is a Lenovo Thinkpad T61, Intel Centrino Pro (dual core), 2GB RAM, 1440×900 screen size, DVD-RW, Intel WiFi/g (see this comment to make it work), bluetooth, integrated SD/Memory Stick/xD/etc card reader, Firewire interface. Much better. Details on Smolt.

I am writing this while Fedora 8 (including some Livna packages) is being installed over the network, as you can see.

Please Encrypt Your BitTorrent Client !

Some evil ISPs implement traffic shaping to specifically limit BitTorrent bandwidth. This brings slower downloads to you and your peer.

All modern BitTorrent clients have encryption capabilities but most of them came disabled by default.

Please configure your client to encrypt connections. You will have faster downloads even if your ISP does not limit your traffic, but because your peer’s ISP does that.

By the way, today I started to use the ultimate BitTorrent client for Linux, BSD and UNIX: rTorrent. It is text and console-based, superlight, superfast, runs very well in my K7/192MB machine, accessible from anywhere through SSH and a screen window.

To install it:

Debian: bash$ sudo apt-get install rtorrent
Fedora/Red Hat: bash$ sudo yum -y install rtorrent

To enable encription I added this line to ~/.rtorrent.rc:

encryption=allow_incoming,try_outgoing,enable_retry

Mobile Codes for Increased Productivity

Man, this is the coolest thing I discovered this month.

Before what I am going to show you now, the only ways to transfer data from your computer to your mobile device was to send an e-mail to yourself and read it in your cell, or through bluetooth, or infrared.

There is a new thing (for me) called Mobile Codes (or QR Codes according to Wikipedia, or Datamatrix) that combined with your mobile’s camera and simple software, will let you actually read the content from your computer screen.

OK, lets explain it with some images to make the concept more user-friendly. Look at the following image.

Think Open :: Think Linux

Yes, I know the image says nothing to you, but your camara-equiped mobile device is able to read “Think Open :: Think Linux”.

We can play with more stuff. The URL for this blog post is here (with a note attached):

http://avi.alkalay.net/2007/11/mobile-codes.html

Lets add some semantics. You can directly call an international number as 00551112345678 using the following barcode:

qrcode

Or SMS “Hi, I just discovered this barcodes that will make my life easier” to same number with this code:

qrcode

The Details

Some modern Nokia phones already come with the software needed to decode this datamatrix. For my Nokia E61i to work, I installed on it a free (registration required) program called Kaywa Reader (their site doesn’t list my model so I selected E65 and it worked fine). So I just shoot the software, point my mobile’s camera to the datamatrix and it instantly decodes the content.

To generate this images, I used the QR Code Generator by Winksite and the Nokia mobile code generator. In fact it all started when I found the Nokia Mobile Code website, almost by an accident.

Oh, and there is a handy Mobile Barcoder Firefox plugin to help me transfer URLs from browser to cellphone.

Install in your cellphone and have fun.