I thought that I would have had at least some progress by now, but I've found that a busy life leaves little time for side projects. Still, this is far from over, even if I am still in the planning stages.
This project is going to be well done, but low budget. I'm going to attempt to code the TMS itself using Python. With its large standard library, almost everything that I'm going to need is already there. It even has support for a database already included, and extensive support for Unicode.
For any images, I plan on using GIMP, a free alternative for graphics design and manipulation. I might even use Inkskape if I decide to use some vector graphics.
I know that this is going to be a fairly large project. I may want to do small chunks of functionality, then go for the larger stuff.
To be continued.
Tuesday, May 6, 2014
Friday, April 11, 2014
"The Project" Part One: The Beginning
Recently, I decided that I needed to hone my programming skills, so I've decided to take it upon myself to create a translation management system (TMS). There are so many translation tools out there, and I'm going to want this one to stand out among them. Right now, it's still in the planning phase (as in I'm planning to do it, but I'm still trying to work out where to begin). So far, I've had a couple of suggestions, but any more would be helpful. Any suggestions?
Ideas so far (mine and others'):
Do you have any other ideas? Leave them in the comments!
Ideas so far (mine and others'):
- An updated design/user interface.
- Purchase Order (PO) creation, or a way to access the information in order to create POs.
- Very customizable filters for XML and zipped XML formats, as well as regular expression filters for plain text formats, and filter chains.
- Customizable locales.
- Customizable segmentation.
Do you have any other ideas? Leave them in the comments!
Saturday, March 29, 2014
The Future: My Thoughts
The other day, I was at a fast food restaurant to... get food, fast. I noticed that their value menu, formerly their dollar menu, had changed since the last time I had been there. The main thing that I noticed, since a side benefit to fast food is also that it's cheap, is that one of my favorite sandwiches, that had been $1.00 for years, had gone up by about $0.30.
Now, like just about everyone, I'd rather not pay more for something. I do understand, though. Inflation. Cost of ingredients. Making money. (As a side note, I find it very odd when someone complains that all a business or employee wants to do is make money. To a degree, that's true for every business, every profession. It may not be their only goal, but if they didn't make money, they would go out of business).
What do my food cravings have to do with localization? Well, it's not just the food service industry that follows this trend. No matter the business, and whether it deals in goods, services, or both, prices go up with the times. People bemoan the rising price of gas. Others probably compare the price of a haircut now to what it was when they were young. Sure, everyone wants to pay less, but the reality of economy is that things tend to get more expensive.
Enter the localization industry. Like every industry, they want to make money. That's no secret. As I mentioned in my side note above, everyone does. Unlike other industries, though, and this is according to what I've observed in several companies and not one in particular, they have a funny way of showing it. Clients who want translation or localization done, who don't understand what a mentally taxing job translation can be, who don't seem to understand sometimes that translation isn't just writing something down in another language, demand better, faster, cheaper translations. Who can blame them? I want cheaper gas. I want a cheaper fast food sandwich (and I wouldn't mind if it were better or came faster, either). Unlike other industries, though, where eventually prices rise, prices have fallen. Deadlines have gotten shorter. Quality expectations have gone up. It sounds pretty good from a client's perspective.
Enter the translators. They, like everyone else, want to make money. They don't mind (to a point past which it becomes practically impossible) to give you better quality faster, but they want more money for their work. Who can blame them? Inflation. Higher levels of stress and mental taxation. All valid reasons to ask for a higher rate.
Enter the language service providers. They are the middle men, and absolutely necessary in the world of translation and localization. They may hire full-time translators, but most of what they do is see that your translation gets done. They usually have a vast network of freelance translators that they turn to, so that you, as the client, don't have to find an individual translator for each language every time. They maintain glossaries and translation memories, to help make the work more efficient and consistent in quality.
And they want to make money. They need to charge more than their freelance translator charges. They need to cover their costs and keep their employees happy. And they have a lot of pressure coming from both sides. I'm sure that this happens in more industries than translation and localization, but this is what I know. The clients want more, better, faster, cheaper. The translator can, to a point, give you more, better, faster, but they want more money for it. The client threatens to go to someone else if you can't give them a better price. The translator does the same. The middle man tries to manage the expectations of both, but eventually gives, raising the rates for the translator, and lowering them for the client.
How do they survive? By juicing every bit of efficiency out of existing technologies. Translation memories can eventually provide big savings. Once you've translated a sentence, you don't need to translate it again. Machine translation is the way of the future, they say, and it can provide some time and money savings if used correctly. (More on machine translation, how it works, and what its limitations are, is planned for a future blog post). But the technologies have their limits. Something's going to give.
Here's what I think is going to happen. Eventually, something is going to give, maybe not all at once, and not across all parts of the industry, but something is going to give. Eventually, language service providers, the essential middle men, will implode, or at least feel enough pressure. They'll realize that translators still need to eat, so they won't push quite as much for lower rates. They'll still make as much use as they can of available technologies, and hopefully work to improve and add to those technologies. And clients will realize that quality translation is essential. They won't like it, and who can blame them, but they'll start to accept some higher rates. An equilibrium will be reached, until the point is pushed again and the cycle continues anew.
Now, like just about everyone, I'd rather not pay more for something. I do understand, though. Inflation. Cost of ingredients. Making money. (As a side note, I find it very odd when someone complains that all a business or employee wants to do is make money. To a degree, that's true for every business, every profession. It may not be their only goal, but if they didn't make money, they would go out of business).
What do my food cravings have to do with localization? Well, it's not just the food service industry that follows this trend. No matter the business, and whether it deals in goods, services, or both, prices go up with the times. People bemoan the rising price of gas. Others probably compare the price of a haircut now to what it was when they were young. Sure, everyone wants to pay less, but the reality of economy is that things tend to get more expensive.
Enter the localization industry. Like every industry, they want to make money. That's no secret. As I mentioned in my side note above, everyone does. Unlike other industries, though, and this is according to what I've observed in several companies and not one in particular, they have a funny way of showing it. Clients who want translation or localization done, who don't understand what a mentally taxing job translation can be, who don't seem to understand sometimes that translation isn't just writing something down in another language, demand better, faster, cheaper translations. Who can blame them? I want cheaper gas. I want a cheaper fast food sandwich (and I wouldn't mind if it were better or came faster, either). Unlike other industries, though, where eventually prices rise, prices have fallen. Deadlines have gotten shorter. Quality expectations have gone up. It sounds pretty good from a client's perspective.
Enter the translators. They, like everyone else, want to make money. They don't mind (to a point past which it becomes practically impossible) to give you better quality faster, but they want more money for their work. Who can blame them? Inflation. Higher levels of stress and mental taxation. All valid reasons to ask for a higher rate.
Enter the language service providers. They are the middle men, and absolutely necessary in the world of translation and localization. They may hire full-time translators, but most of what they do is see that your translation gets done. They usually have a vast network of freelance translators that they turn to, so that you, as the client, don't have to find an individual translator for each language every time. They maintain glossaries and translation memories, to help make the work more efficient and consistent in quality.
And they want to make money. They need to charge more than their freelance translator charges. They need to cover their costs and keep their employees happy. And they have a lot of pressure coming from both sides. I'm sure that this happens in more industries than translation and localization, but this is what I know. The clients want more, better, faster, cheaper. The translator can, to a point, give you more, better, faster, but they want more money for it. The client threatens to go to someone else if you can't give them a better price. The translator does the same. The middle man tries to manage the expectations of both, but eventually gives, raising the rates for the translator, and lowering them for the client.
How do they survive? By juicing every bit of efficiency out of existing technologies. Translation memories can eventually provide big savings. Once you've translated a sentence, you don't need to translate it again. Machine translation is the way of the future, they say, and it can provide some time and money savings if used correctly. (More on machine translation, how it works, and what its limitations are, is planned for a future blog post). But the technologies have their limits. Something's going to give.
Here's what I think is going to happen. Eventually, something is going to give, maybe not all at once, and not across all parts of the industry, but something is going to give. Eventually, language service providers, the essential middle men, will implode, or at least feel enough pressure. They'll realize that translators still need to eat, so they won't push quite as much for lower rates. They'll still make as much use as they can of available technologies, and hopefully work to improve and add to those technologies. And clients will realize that quality translation is essential. They won't like it, and who can blame them, but they'll start to accept some higher rates. An equilibrium will be reached, until the point is pushed again and the cycle continues anew.
Monday, March 24, 2014
Why is this here?
Hello all,
Someone once told me that an expert is someone who uses jargon without realizing that they're doing it. I don't really consider myself an expert in the field of localization, but I might just fit that definition. I find myself talking about alignments, TM updates, segmentation, concatenation, character encodings and formatting, and I'm sure that some people look at me like I'm speaking a foreign language. Still, I don't know everything, and I don't know that I ever will.
I do see things, though, and I want to share some of them. This is the place that I've chosen to do that. I know that, as of writing this, this blog only has two followers, one comment, total, but who knows? Maybe someday someone will see something that will help them. Until then, I'll just keep doing what I'm doing.
My rough schedule for updating this blog is the first and third Saturdays of the month, which makes this post over a week behind. The next scheduled blog post, which should happen on April 5, will be another tool review. I'm planning on making a special post, though, this coming Saturday, about the trends that I've seen in the industry and my thoughts on those trends.
Until then, keep localizing!
-Dale
Someone once told me that an expert is someone who uses jargon without realizing that they're doing it. I don't really consider myself an expert in the field of localization, but I might just fit that definition. I find myself talking about alignments, TM updates, segmentation, concatenation, character encodings and formatting, and I'm sure that some people look at me like I'm speaking a foreign language. Still, I don't know everything, and I don't know that I ever will.
I do see things, though, and I want to share some of them. This is the place that I've chosen to do that. I know that, as of writing this, this blog only has two followers, one comment, total, but who knows? Maybe someday someone will see something that will help them. Until then, I'll just keep doing what I'm doing.
My rough schedule for updating this blog is the first and third Saturdays of the month, which makes this post over a week behind. The next scheduled blog post, which should happen on April 5, will be another tool review. I'm planning on making a special post, though, this coming Saturday, about the trends that I've seen in the industry and my thoughts on those trends.
Until then, keep localizing!
-Dale
Monday, March 3, 2014
Tool Spotlight: PortableApps
Occasionally, I run into a program or a set of programs that I just want to share with the world. Today, I'm going to talk about PortableApps.
I first ran into PortableApps several years ago, when I purchased a USB drive. My thought process was, essentially, like this:
"Gee, I wonder if I can set this up to be like a portable computer..."
Enter PortableApps, one of the results that came up when I was looking through my favorite search engine. As long as you're on a Windows computer, and you have access to your USB drive, you can have a set of programs that you can use.
PortableApps itself acts kind of like a Start menu. It shows up with the rest of your running background programs, by default on the lower right of your screen, along with your network connection, volume, etc. You click on it, and it allows you to launch any programs on your USB drive, as long as they follow a pretty basic directory structure.
While this doesn't seem like much, and might not even be much if you have a tablet that you can use as a computer, it means that you can borrow someone else's computer and use it like your own. If you're a hobbyist developer, you can put all of your projects, development environments, etc. on your thumb drive and not worry about a different computer having a different version of that environment. If you like listening to music and want to take that music to work with you, set your PortableApps environment up with a media player, stick your music on there, and have it to go.
Again, you may not fit into this category, but even if you don't, remember the link to the PortableApps website. Besides this nifty environment they offer, they have a pretty sizable library of free apps. All of the apps that they have are free. And you can install them anywhere. And since they're designed to work from a thumb drive, they install to one folder. Since they don't modify any system files at all, you shouldn't even need installation privileges. To uninstall, simply delete the folder. So if you quickly need a photo editing program, they've got you covered. If you need a little widget that you only need once a month or less, just download it, use it, and get rid of it.
So, yeah, use PortableApps. Discover what you can do with it, and what it can do for you.
I first ran into PortableApps several years ago, when I purchased a USB drive. My thought process was, essentially, like this:
"Gee, I wonder if I can set this up to be like a portable computer..."
Enter PortableApps, one of the results that came up when I was looking through my favorite search engine. As long as you're on a Windows computer, and you have access to your USB drive, you can have a set of programs that you can use.
PortableApps itself acts kind of like a Start menu. It shows up with the rest of your running background programs, by default on the lower right of your screen, along with your network connection, volume, etc. You click on it, and it allows you to launch any programs on your USB drive, as long as they follow a pretty basic directory structure.
While this doesn't seem like much, and might not even be much if you have a tablet that you can use as a computer, it means that you can borrow someone else's computer and use it like your own. If you're a hobbyist developer, you can put all of your projects, development environments, etc. on your thumb drive and not worry about a different computer having a different version of that environment. If you like listening to music and want to take that music to work with you, set your PortableApps environment up with a media player, stick your music on there, and have it to go.
Again, you may not fit into this category, but even if you don't, remember the link to the PortableApps website. Besides this nifty environment they offer, they have a pretty sizable library of free apps. All of the apps that they have are free. And you can install them anywhere. And since they're designed to work from a thumb drive, they install to one folder. Since they don't modify any system files at all, you shouldn't even need installation privileges. To uninstall, simply delete the folder. So if you quickly need a photo editing program, they've got you covered. If you need a little widget that you only need once a month or less, just download it, use it, and get rid of it.
So, yeah, use PortableApps. Discover what you can do with it, and what it can do for you.
Tuesday, April 30, 2013
Using SDL's WinAlign to Align InDesign Files
If you have InDesign files in two locales and you want to align them for a translation memory, there are a couple of ways that you can do this.
- If you happen to have the version of InDesign that your files were created in, and InDesign CS4, you can save it down first to IDML, then to INX. WinAlign has a selection for INX files.
- Choose File -> Export for -> Dreamweaver. Then you can align the files using the HTML option. Just make sure you choose the same export options. There might be some parts that don't export in the same order, but overall, it works pretty well.
Monday, November 19, 2012
Transfering Your iTunes Library from OSX to Windows 8
This post is not about localization, but it is about something that I was able to do because of my experience as a localization engineer, manipulating files and editing XML.
We recently purchased a desktop with the Windows 8 operating system, in part to replace her pre-Intel iMac.
Now, there are guides to transfer your iTunes library from Mac to Mac. There are guides to transfer your library from Windows to Mac. But, as far as I can tell, there are no guides to transfer your library from Mac to Windows.
I was able to do it, without losing any information about the playlists or play count, etc.* This is how.
This guide was inspired by this guide and this guide, but mostly the first one.
Follow parts 1 and 2 in that first link. This will get all of your music and videos, as well as the database and XML files that contain the information about your library, and put them on an external drive. I used an external hard drive that connects via USB.
Step 2: Make sure iTunes is set up on your new computer.
Download iTunes and open it. Then close it. That's it.
Step 3: Back up the XML file from your new computer.
In your iTunes folder, on your new computer, locate "iTunes Music Library.xml". Copy it somewhere, like your desktop.
Step 4: Rename your old iTunes music folder, inside the iTunes folder to match the new one.
Go to your external hard drive and change the name of your music folder, if needed. On our new computer, it's "iTunes Media". It was something like "iTunes Music Library" or something like that before.
Step 5: Get the file extensions right.
First, make sure to turn on viewing file extensions. Then, locate 3 files inside your old iTunes folder. They will be named "iTunes Library Extras.itdb", "iTunes Library Genius.itdb" and "iTunes Library.itl". On our Mac, one of them was missing the file extension. Add it if needed.
Step 6: Copy the files over.
Copy your files from your external drive to your new computer. Respond "Yes" to any prompts you may get to replace files.
Step 7: Edit the XML.
Do not get overwhelmed. That XML just contains all of the information about your library. No big deal. (Besides, if you mess it up, you have the old one backed up on your old computer and your external hard drive. You backed up your new one in Step 3).
To do this, open a text editor. Not Notepad. Anything but Notepad. Mac uses a different way to represent a new line, and Notepad will display everything in one line. Use Notepad++, or copy the text into Google Docs, or something. Compare the first several lines between your new one (that you backed up in Step 3) and your old one. There are two lines that are important to change in your old one:
<key>Application Version</key><string>x.x</string>
Change this to the version of iTunes in your new XML file. Our version is 10.7
<key>Music Folder</key><string>file://localhost/<file Location></string>
Your old one will have a Mac location. Copy the line from your new one and paste it to replace your old one.
Everything from "Major Version" to "Music Folder" should match the new, backed-up file. Don't change anything below or above that. After you're done editing, save.
Step 8: Open iTunes.
You're done!
* Full disclosure: There were 8 tracks that did not transfer, because apparently the path was too long. They were all from the same album - an audio book. We'll manually transfer them over later. Still, 8 tracks out of almost 2,000 isn't bad. Also, this guide should work for transferring from Windows to Windows as well.
We recently purchased a desktop with the Windows 8 operating system, in part to replace her pre-Intel iMac.
Now, there are guides to transfer your iTunes library from Mac to Mac. There are guides to transfer your library from Windows to Mac. But, as far as I can tell, there are no guides to transfer your library from Mac to Windows.
I was able to do it, without losing any information about the playlists or play count, etc.* This is how.
This guide was inspired by this guide and this guide, but mostly the first one.
Dale's Guide to Transferring your iTunes Library
Step 1: Transfer your files.Follow parts 1 and 2 in that first link. This will get all of your music and videos, as well as the database and XML files that contain the information about your library, and put them on an external drive. I used an external hard drive that connects via USB.
Step 2: Make sure iTunes is set up on your new computer.
Download iTunes and open it. Then close it. That's it.
Step 3: Back up the XML file from your new computer.
In your iTunes folder, on your new computer, locate "iTunes Music Library.xml". Copy it somewhere, like your desktop.
Step 4: Rename your old iTunes music folder, inside the iTunes folder to match the new one.
Go to your external hard drive and change the name of your music folder, if needed. On our new computer, it's "iTunes Media". It was something like "iTunes Music Library" or something like that before.
Step 5: Get the file extensions right.
First, make sure to turn on viewing file extensions. Then, locate 3 files inside your old iTunes folder. They will be named "iTunes Library Extras.itdb", "iTunes Library Genius.itdb" and "iTunes Library.itl". On our Mac, one of them was missing the file extension. Add it if needed.
Step 6: Copy the files over.
Copy your files from your external drive to your new computer. Respond "Yes" to any prompts you may get to replace files.
Step 7: Edit the XML.
Do not get overwhelmed. That XML just contains all of the information about your library. No big deal. (Besides, if you mess it up, you have the old one backed up on your old computer and your external hard drive. You backed up your new one in Step 3).
To do this, open a text editor. Not Notepad. Anything but Notepad. Mac uses a different way to represent a new line, and Notepad will display everything in one line. Use Notepad++, or copy the text into Google Docs, or something. Compare the first several lines between your new one (that you backed up in Step 3) and your old one. There are two lines that are important to change in your old one:
<key>Application Version</key><string>x.x</string>
Change this to the version of iTunes in your new XML file. Our version is 10.7
<key>Music Folder</key><string>file://localhost/<file Location></string>
Your old one will have a Mac location. Copy the line from your new one and paste it to replace your old one.
Everything from "Major Version" to "Music Folder" should match the new, backed-up file. Don't change anything below or above that. After you're done editing, save.
Step 8: Open iTunes.
You're done!
* Full disclosure: There were 8 tracks that did not transfer, because apparently the path was too long. They were all from the same album - an audio book. We'll manually transfer them over later. Still, 8 tracks out of almost 2,000 isn't bad. Also, this guide should work for transferring from Windows to Windows as well.
Saturday, October 27, 2012
Source Files
As localization professionals, we appreciate it when our clients are considerate of what we need to better serve them. One of the ways that they can show this consideration is by providing source files for whatever they want translated.
What are source files? They are files that are saved locally to our drives that can be easily edited either in their native program or in the translation memory tool that we choose to use.
Some examples of non-source files:
Some examples of source files:
Non-source files are meant for viewing. There's nothing wrong with them, but they are not meant to be edited or localized. Source files are meant for editing and generating the non-source files. They are what we need in order to localize content in the most effective manner possible.
What are source files? They are files that are saved locally to our drives that can be easily edited either in their native program or in the translation memory tool that we choose to use.
Some examples of non-source files:
- Images (GIF, JPEG, PNG...)
- PDF files
- Web site URLs (or YouTube URLs for videos)
Some examples of source files:
- Source image files (Photoshop, Adobe Illustrator)
- DTP source files (Adobe Illustrator, Adobe InDesign, Quark)
- Source web files (HTML, PHP)
Non-source files are meant for viewing. There's nothing wrong with them, but they are not meant to be edited or localized. Source files are meant for editing and generating the non-source files. They are what we need in order to localize content in the most effective manner possible.
Saturday, April 7, 2012
Holidays
After more than 5 months of absence, I'm back with a renewed determination to share what interesting tidbits of information about localization I can with the world.
This weekend, as many of you know, is Easter weekend. In the US, it generally means Easter eggs, the Easter bunny, etc. In several other countries, however, it means at least a three-day weekend, as many predominantly Christian countries have Good Friday as a holiday. Some also have Thursday and/or Monday off.
What does that mean to the localization professional? Well, while localization is more than translation, translation is still a key component of localization. If you take translation out of the equation, you're out of luck. And if you try to find a freelance translator in a country where a holiday is widely celebrated by doing anything but working... you may be out of luck.
It's key to let your clients know, as soon as you do, when international holidays are coming up that may delay delivery time for some of their usual languages. It's also key to communicate with any international resources that you may work with, to see which major holidays they have that might affect their availability. As always, it's important to maintain reasonable expectations, and to set reasonable expectations for your clients.
This weekend, as many of you know, is Easter weekend. In the US, it generally means Easter eggs, the Easter bunny, etc. In several other countries, however, it means at least a three-day weekend, as many predominantly Christian countries have Good Friday as a holiday. Some also have Thursday and/or Monday off.
What does that mean to the localization professional? Well, while localization is more than translation, translation is still a key component of localization. If you take translation out of the equation, you're out of luck. And if you try to find a freelance translator in a country where a holiday is widely celebrated by doing anything but working... you may be out of luck.
It's key to let your clients know, as soon as you do, when international holidays are coming up that may delay delivery time for some of their usual languages. It's also key to communicate with any international resources that you may work with, to see which major holidays they have that might affect their availability. As always, it's important to maintain reasonable expectations, and to set reasonable expectations for your clients.
Saturday, October 29, 2011
Encodings, Part 2
So... Different places started using different encodings to represent the characters in their languages. There are several ways to represent the characters, but none of them would work with each other.
Enter Unicode. Unicode is a largely successful way to represent every character, ever. It's largely what's used in localization. Still, there are some things that you need to keep in mind. There are several ways to represent Unicode.
You might hear the terms "UTF-8," "UTF-16," "Unicode," "Little Endian" or "Big Endian." Even UTF-32 exists now. What's it all mean?
UTF-16 is the most straightforward way to represent Unicode, at least the first, well, 16 bits of it, which is most of the current Unicode set. UTF-32 is the most straightforward way to represent all current Unicode characters, and then some. The endian-ness of these encodings depends (or at least depended) on how computers work internally, which bytes naturally get read first, etc. That's where something called a Byte Order Mark (BOM) comes into play. It's the first two(in the case of UTF-16) or 4(in the case of UTF-32) bytes that lets you know what order the bytes come in, or the "endian-ness." UTF-8 can have a BOM of 3 bytes, but it just lets the computer know that it's UTF-8 and not just ANSI/ASCII (the old stuff that used 8 bits, or one byte, per character).
UTF-8 is the smallest in terms of size, UTF-32 is the largest but most straight forward. The encodings you'll see the most in localization are UTF-8 and UTF-16. If you need more information, let me know in the comments! Or try Google. I've given you plenty of terms to get started. :)
Enter Unicode. Unicode is a largely successful way to represent every character, ever. It's largely what's used in localization. Still, there are some things that you need to keep in mind. There are several ways to represent Unicode.
You might hear the terms "UTF-8," "UTF-16," "Unicode," "Little Endian" or "Big Endian." Even UTF-32 exists now. What's it all mean?
UTF-16 is the most straightforward way to represent Unicode, at least the first, well, 16 bits of it, which is most of the current Unicode set. UTF-32 is the most straightforward way to represent all current Unicode characters, and then some. The endian-ness of these encodings depends (or at least depended) on how computers work internally, which bytes naturally get read first, etc. That's where something called a Byte Order Mark (BOM) comes into play. It's the first two(in the case of UTF-16) or 4(in the case of UTF-32) bytes that lets you know what order the bytes come in, or the "endian-ness." UTF-8 can have a BOM of 3 bytes, but it just lets the computer know that it's UTF-8 and not just ANSI/ASCII (the old stuff that used 8 bits, or one byte, per character).
UTF-8 is the smallest in terms of size, UTF-32 is the largest but most straight forward. The encodings you'll see the most in localization are UTF-8 and UTF-16. If you need more information, let me know in the comments! Or try Google. I've given you plenty of terms to get started. :)
Saturday, October 22, 2011
Encodings, Part 1
What are you reading right now? Letters? Words? Ideas? There really is no wrong answer, unless you happen to be a computer.
If you're a computer, you're reading numbers. Lots and lots of numbers. Everything in computers is numbers, and text is no exception.
How do those numbers turn into text? It all starts out with a font. A font, basically, is a group of small pictures, pictures representing how to draw each character. In the computer, a character is stored as the index to a specific character in the font, or, in other words, which character should be drawn.
Okay, so where do encodings come in, and what are they exactly?
Computers began in the English-speaking world. Apple and Microsoft, the two big players in the computer world, are both American companies. Like a lot of companies that start out in the United States, they didn't really think globally at first. A character was represented by small numbers that would fit within one byte (Computers those days weren't really able to handle anything bigger anyway). One byte can handle the numbers from 0 to 255. That's more than enough for all characters in English ('A' and 'a' being two different characters, etc.), but not nearly enough for some other languages, like, say, Japanese.
This is where things got complicated. And I'll pick it up from here next week.
If you're a computer, you're reading numbers. Lots and lots of numbers. Everything in computers is numbers, and text is no exception.
How do those numbers turn into text? It all starts out with a font. A font, basically, is a group of small pictures, pictures representing how to draw each character. In the computer, a character is stored as the index to a specific character in the font, or, in other words, which character should be drawn.
Okay, so where do encodings come in, and what are they exactly?
Computers began in the English-speaking world. Apple and Microsoft, the two big players in the computer world, are both American companies. Like a lot of companies that start out in the United States, they didn't really think globally at first. A character was represented by small numbers that would fit within one byte (Computers those days weren't really able to handle anything bigger anyway). One byte can handle the numbers from 0 to 255. That's more than enough for all characters in English ('A' and 'a' being two different characters, etc.), but not nearly enough for some other languages, like, say, Japanese.
This is where things got complicated. And I'll pick it up from here next week.
Saturday, October 15, 2011
Translation Memories: Pros and Cons
To say that translation memories are popular in the translation/localization industry would be a gross understatement. Chances are, if you have worked with or in the industry in any way, you have heard "TM," "translation memory" or some other term referring to translation memories.
Almost every translation system or tool that claims to increase productivity uses a translation memory. Some make claims that their memory is somehow better than another tool's, but the basics are the same.
TMs are exactly what their name describes: They are a way for a computer to remember what has been translated before. This is done on a segment level (or on a sentence level, basically, though there are exceptions). If you come across the same segment later, or a similar enough one, your translation tool will be able to look it up in the translation and either offer it up as a suggestion or simply plug it in as the translation.
There are a lot of benefits to having a translation memory, when it is managed correctly. For example, if a client has an update to a file, they might ask you to only translate what has changed and paste it into the previously translated file. If you have a translation memory, you can just stick the whole file into the tool, and it will fill out what has already been translated, leaving you to only translate what has changed. No need to search for the changes, find the corresponding place in the previous translation, paste, etc...
There might also be a lot of occurrences of the same segment in the same document. Translation memories can be updated as you work, so if you get to a segment that you've seen before, the translation memory can fill it in for you. This is also the case with slogans, etc., where the chance of repetition is high. Translate once, then reuse as needed.
With advantages such as these, productivity can and does go up. However, translation memories have their drawbacks, several of which are often ignored. You should know that I'm not making these up, that in about a year of working in the industry as a localization engineer, I've come across each of these drawbacks at least once.
The first drawback that I'll mention has to do with translator laziness or complacency. It's not really the translator's fault, either. It's something of a trap. Most translation memory tools have the option to fill in "fuzzy" (inexact) matches if their degree of similarity to the segment in question is above a certain degree. It's very easy, as a translator, to see that a translation has already been inserted and move on. After all, your time is valuable. However, the meaning of a sentence can change without the sentence changing much at all. Some theoretical examples: "You must do that." vs "You must not do that."; "John has 1 dog." vs "John has 3 dogs."; etc. If a translator misses one change, it gets put into the translation memory. Then you have an incorrect translation all ready to be used again. That's not even mentioning exact matches that may be out of context, so their translations should be different but aren't, because it's not usually considered worth a translator's time to look at something that's already translated at a 100% match.
The second drawback can be seen as an extension of the first, except that this time it's not the translator at all who's at fault, but a side effect of what may have happened previously. That drawback is error propagation. If a segment is translated incorrectly, it will be incorrect all over the place. I've seen this happen literally hundreds of times within a very technical set of files. The worst part? Sometimes it's very difficult to fix. Because the same source segment can be translated different ways given different contexts, some translation memory tools will save any changes as a different entry into the translation memory. Depending on the features of the tool, the incorrect translation may pop up again. Sure, you can look in the memory itself and delete the offending segment, but translation memories can get big, so it can be difficult to manage them sometimes.
The third drawback (and the last that I'm going to mention) is unrealistic expectations. While this is not really the fault of translation memories, it effects people who use them. I offer the following examples, though there are a lot of ways expectations can be unrealistic. Example #1: Almost all translation memories work on a segment level. They don't deal with anything smaller than that. A lot of companies will charge/pay depending on the degree of matches in a translation memory. Why pay for something that's already translated, right? Well, some people seem to think that if they change "a few words" in some sentences, they will only be charged a lower match rate on those words. However, because translation memories work at a segment level, if the match rate (how much of the segment is similar to the entry in the translation memory) is low enough that it passes the threshold of what is charged, it affects every word in that segment, not just the words that have been changed. I've seen a few clients get into arguments because "only a few words" had been changed. Example #2: Turnaround times. Some clients, and I'm talking both about the end client and the translation companies that use freelancers, seem to think that because you have a translation memory tool, all translations can get done lightening fast, regardless of how repetitive the text is, or how long it is, or if you've ever translated anything like it before (i.e. if you have segments in your translation memory from similar texts). Translation is still work, and sometimes, regardless of what tools you're using, it takes time.
So translation memories can be huge time savers. They can make sure that your wording is consistent in similar segments, and they can give suggestions based on previous translations. However, they are not miracle workers and in some cases can make things worse. Missing fuzzy matches can allow errors to sneak in, and sometimes errors can be propagated to other sections, or even completely different documents. You have to be careful when using a translation memory, and you have to make sure to manage clients' expectations.
Almost every translation system or tool that claims to increase productivity uses a translation memory. Some make claims that their memory is somehow better than another tool's, but the basics are the same.
TMs are exactly what their name describes: They are a way for a computer to remember what has been translated before. This is done on a segment level (or on a sentence level, basically, though there are exceptions). If you come across the same segment later, or a similar enough one, your translation tool will be able to look it up in the translation and either offer it up as a suggestion or simply plug it in as the translation.
There are a lot of benefits to having a translation memory, when it is managed correctly. For example, if a client has an update to a file, they might ask you to only translate what has changed and paste it into the previously translated file. If you have a translation memory, you can just stick the whole file into the tool, and it will fill out what has already been translated, leaving you to only translate what has changed. No need to search for the changes, find the corresponding place in the previous translation, paste, etc...
There might also be a lot of occurrences of the same segment in the same document. Translation memories can be updated as you work, so if you get to a segment that you've seen before, the translation memory can fill it in for you. This is also the case with slogans, etc., where the chance of repetition is high. Translate once, then reuse as needed.
With advantages such as these, productivity can and does go up. However, translation memories have their drawbacks, several of which are often ignored. You should know that I'm not making these up, that in about a year of working in the industry as a localization engineer, I've come across each of these drawbacks at least once.
The first drawback that I'll mention has to do with translator laziness or complacency. It's not really the translator's fault, either. It's something of a trap. Most translation memory tools have the option to fill in "fuzzy" (inexact) matches if their degree of similarity to the segment in question is above a certain degree. It's very easy, as a translator, to see that a translation has already been inserted and move on. After all, your time is valuable. However, the meaning of a sentence can change without the sentence changing much at all. Some theoretical examples: "You must do that." vs "You must not do that."; "John has 1 dog." vs "John has 3 dogs."; etc. If a translator misses one change, it gets put into the translation memory. Then you have an incorrect translation all ready to be used again. That's not even mentioning exact matches that may be out of context, so their translations should be different but aren't, because it's not usually considered worth a translator's time to look at something that's already translated at a 100% match.
The second drawback can be seen as an extension of the first, except that this time it's not the translator at all who's at fault, but a side effect of what may have happened previously. That drawback is error propagation. If a segment is translated incorrectly, it will be incorrect all over the place. I've seen this happen literally hundreds of times within a very technical set of files. The worst part? Sometimes it's very difficult to fix. Because the same source segment can be translated different ways given different contexts, some translation memory tools will save any changes as a different entry into the translation memory. Depending on the features of the tool, the incorrect translation may pop up again. Sure, you can look in the memory itself and delete the offending segment, but translation memories can get big, so it can be difficult to manage them sometimes.
The third drawback (and the last that I'm going to mention) is unrealistic expectations. While this is not really the fault of translation memories, it effects people who use them. I offer the following examples, though there are a lot of ways expectations can be unrealistic. Example #1: Almost all translation memories work on a segment level. They don't deal with anything smaller than that. A lot of companies will charge/pay depending on the degree of matches in a translation memory. Why pay for something that's already translated, right? Well, some people seem to think that if they change "a few words" in some sentences, they will only be charged a lower match rate on those words. However, because translation memories work at a segment level, if the match rate (how much of the segment is similar to the entry in the translation memory) is low enough that it passes the threshold of what is charged, it affects every word in that segment, not just the words that have been changed. I've seen a few clients get into arguments because "only a few words" had been changed. Example #2: Turnaround times. Some clients, and I'm talking both about the end client and the translation companies that use freelancers, seem to think that because you have a translation memory tool, all translations can get done lightening fast, regardless of how repetitive the text is, or how long it is, or if you've ever translated anything like it before (i.e. if you have segments in your translation memory from similar texts). Translation is still work, and sometimes, regardless of what tools you're using, it takes time.
So translation memories can be huge time savers. They can make sure that your wording is consistent in similar segments, and they can give suggestions based on previous translations. However, they are not miracle workers and in some cases can make things worse. Missing fuzzy matches can allow errors to sneak in, and sometimes errors can be propagated to other sections, or even completely different documents. You have to be careful when using a translation memory, and you have to make sure to manage clients' expectations.
Saturday, October 8, 2011
Concatenation is evil.
While I was a student, I participated in a group project to localize a web application. We had to find our own client and everything. We found a nice client with a cool application, and they made things very straightforward for us. They gave us a list of strings that was easily prepared for translation.
There was one problem, though. String concatenation. There was a lot of it. What is string concatenation? It's when you have two strings and you stick them together. One common example: "You have " + (some number) + " item(s) in your shopping cart." The more there is of this, the more difficult it is to localize.
String concatenation occurs almost exclusively in software and websites. In fact, I can't think of any exceptions. It's really easy for programmers (or developers, or whatever they want to call themselves) to fall into the trap of string concatenation. In the example listed above, the number would be calculated using some code somewhere in the application. The easiest way to put the number into the sentence is concatenation. So why does concatenation pose such a problem to translation and localization?
First, let me briefly mention internal strings vs. external strings. Internal strings are strings that are put directly into the code. This might be in the JavaScript of a website or in the computer code of a program. Depending on the complexity of the application, internal strings are almost impossible to localize. It can be done, but it takes longer and costs more. Why? Because you have to sort through all of the code, decide what is relevant for translation and hope that you don't mess up the code in the meantime.
Most people who want their applications localized know this. If not, they will soon. So they externalize their strings. What does this mean? It means you get some sort of list of (identifier) + (string).
This is why concatenation is a problem. You may find something like this:
string.beginning = "You have "
string.end = " item(s) in your shopping cart."
If you speak another language, you may already see the problem. If not, here's a basic explanation. Different languages have different grammar rules. Sometimes, this affects word order. Some language's grammar rules might force the sentence's structure to be "Your shopping cart contains (number) item(s)." or something like that. Other languages might be different. You could, in theory, translate "You have " as "Your shopping cart contains ", but that has its own problems. For one thing, string concatenation often comes paired with string reuse. "You have " might be used somewhere else in the program. It would also pollute your translation memory (more on that in a different post in the future). The list might also look like this:
string.1 = "You have "
string.2 = "something completely unrelated to the contents of your shopping cart"
...
string.453 = "item(s) in your shopping cart."
In this case, context is completely lost. Sometimes the strings might not even appear in the right order in the list. So how do we avoid problems with string concatenation? The answer is simple: We somehow convince our clients to avoid string concatenation. Avoid it like you would avoid tall trees during a thunderstorm.
Unfortunately, getting rid of string concatenation will also get rid of string reuse. That's a nightmare to localization, too, so it's alright with us to get rid of it, but string reuse is popular among some programmers, because they don't have to type as much, and programming is all about making things more efficient. If this is mentioned, you might want to mention that they might need to write completely different code for each language if they still want to use concatenation. That might get their attention.
There are right ways and less right ways to get rid of concatenation. Some strings, like the one mentioned above, have parts that might change. Here is a really wrong way to do it:
string.1 = "You have 1 item in your shopping cart."
string.2 = "You have 2 items in your shopping cart."
...
string.500 = "You have 500 items in your shopping cart."
I think it's obvious what's wrong with this. More strings = more time and money spent by everybody. And it might not always be possible to calculate every possibility.
Here's a less wrong way to do it:
string.itemsInList = "You have %d(numberOfItems) item(s) in your shopping cart."
The above way is a way to cheat string concatenation. You put some sort of placeholder (and the way to do this will change with each computer language, but almost all of them support it) in the middle to keep the string together, and you insert the value later. There are problems with this. It might cause awkward grammar. It might cause confusion. But it's better than concatenation, and it's better than a case for every alternative, especially when it could be an infinitely long list.
Here's what I recommend, where possible:
string.itemsInList = "Number of items in your list: %d(numberOfItems)"
Put the variable part(s) of the string at the end of the string, with a logically complete phrase before it. Or, and this would be a correct usage of string concatenation:
string.itemsInList = "Number of items in your list: "
Then, in the code, the programmer could concatenate that logically complete string with the number.
As you can see, concatenation can cause more than a minor headache for localization teams. It should be managed or avoided, when possible.
Feel free to share your comments below. This is a new blog, so feel free to include things that you would like addressed. I am by no means an expert on everything in localization, but I'm willing to share what knowledge I have and observations that I have made.
There was one problem, though. String concatenation. There was a lot of it. What is string concatenation? It's when you have two strings and you stick them together. One common example: "You have " + (some number) + " item(s) in your shopping cart." The more there is of this, the more difficult it is to localize.
String concatenation occurs almost exclusively in software and websites. In fact, I can't think of any exceptions. It's really easy for programmers (or developers, or whatever they want to call themselves) to fall into the trap of string concatenation. In the example listed above, the number would be calculated using some code somewhere in the application. The easiest way to put the number into the sentence is concatenation. So why does concatenation pose such a problem to translation and localization?
First, let me briefly mention internal strings vs. external strings. Internal strings are strings that are put directly into the code. This might be in the JavaScript of a website or in the computer code of a program. Depending on the complexity of the application, internal strings are almost impossible to localize. It can be done, but it takes longer and costs more. Why? Because you have to sort through all of the code, decide what is relevant for translation and hope that you don't mess up the code in the meantime.
Most people who want their applications localized know this. If not, they will soon. So they externalize their strings. What does this mean? It means you get some sort of list of (identifier) + (string).
This is why concatenation is a problem. You may find something like this:
string.beginning = "You have "
string.end = " item(s) in your shopping cart."
If you speak another language, you may already see the problem. If not, here's a basic explanation. Different languages have different grammar rules. Sometimes, this affects word order. Some language's grammar rules might force the sentence's structure to be "Your shopping cart contains (number) item(s)." or something like that. Other languages might be different. You could, in theory, translate "You have " as "Your shopping cart contains ", but that has its own problems. For one thing, string concatenation often comes paired with string reuse. "You have " might be used somewhere else in the program. It would also pollute your translation memory (more on that in a different post in the future). The list might also look like this:
string.1 = "You have "
string.2 = "something completely unrelated to the contents of your shopping cart"
...
string.453 = "item(s) in your shopping cart."
In this case, context is completely lost. Sometimes the strings might not even appear in the right order in the list. So how do we avoid problems with string concatenation? The answer is simple: We somehow convince our clients to avoid string concatenation. Avoid it like you would avoid tall trees during a thunderstorm.
Unfortunately, getting rid of string concatenation will also get rid of string reuse. That's a nightmare to localization, too, so it's alright with us to get rid of it, but string reuse is popular among some programmers, because they don't have to type as much, and programming is all about making things more efficient. If this is mentioned, you might want to mention that they might need to write completely different code for each language if they still want to use concatenation. That might get their attention.
There are right ways and less right ways to get rid of concatenation. Some strings, like the one mentioned above, have parts that might change. Here is a really wrong way to do it:
string.1 = "You have 1 item in your shopping cart."
string.2 = "You have 2 items in your shopping cart."
...
string.500 = "You have 500 items in your shopping cart."
I think it's obvious what's wrong with this. More strings = more time and money spent by everybody. And it might not always be possible to calculate every possibility.
Here's a less wrong way to do it:
string.itemsInList = "You have %d(numberOfItems) item(s) in your shopping cart."
The above way is a way to cheat string concatenation. You put some sort of placeholder (and the way to do this will change with each computer language, but almost all of them support it) in the middle to keep the string together, and you insert the value later. There are problems with this. It might cause awkward grammar. It might cause confusion. But it's better than concatenation, and it's better than a case for every alternative, especially when it could be an infinitely long list.
Here's what I recommend, where possible:
string.itemsInList = "Number of items in your list: %d(numberOfItems)"
Put the variable part(s) of the string at the end of the string, with a logically complete phrase before it. Or, and this would be a correct usage of string concatenation:
string.itemsInList = "Number of items in your list: "
Then, in the code, the programmer could concatenate that logically complete string with the number.
As you can see, concatenation can cause more than a minor headache for localization teams. It should be managed or avoided, when possible.
Feel free to share your comments below. This is a new blog, so feel free to include things that you would like addressed. I am by no means an expert on everything in localization, but I'm willing to share what knowledge I have and observations that I have made.
Subscribe to:
Posts (Atom)