Pages in topic:   < [1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27] >
Netflix Hermes test
Thread poster: Magdalena Adamus
Butze
Butze
Italy
German to Italian
I don't agree Apr 29, 2017

Sosey wrote:


And yeah, idioms are definitely important, but only if you place them inside a translation job. If you already know they are idioms, all you have to do is google them. That's why I don't think those first three phases were enough to properly evaluate a candidate.
Idioms and everything else we did in the first three sections were important, but only if you also check the subtitle part. The test wouldn't be complete and accurate without any of those phases.


This is audiovisual translation. It's all about the spoken language, it's idioms and puns and jokes and you cannot excell at it if you don't know exactly what you are talking about. Of all the different types of translation, I think audiovisual is the one which requires the widest knowledge of the source language, if you want to be fast, cost-effective and give a high-quality service.
My experience is that many choose to do audiovisual to top up, but they come from literary translation, for example. And yeah, their translations are grammatically perfect, but they wouldn't spot an idiom if their life depended on it.
I think Netflix has all the right to give priority to this aspect. Then we both agree on the fact that a more comprehensive evaluation would have been preferable.


 
Franjo Varšić
Franjo Varšić
Croatia
Tutorial? Apr 29, 2017

maroscz wrote:

First link to the tutorial page comes when you register for the test, before you get the SMS. Then there is the link on the Hermes homepage.

In my test I got 2 questions where there was a typo (wrong letter). I marked one example as having no errors. All the other examples clearly breached some of the timing rules stated in this document.

https://backlothelp.netflix.com/hc/en-us/articles/215758617-Timed-Text-Style-Guide-General-Requirements

I don't have Netflix subscription, but from what i read in recent weeks, the bad quality of the subtitles was the problem, that lead Netflix to change the way they handle the task, and made them come up with the Hermes platform.

Unfortunately it seems, that while they were trying to get rid of the subpar vendors, they chose a subpar vendor for the task.


When you say tutorial, you're referring to something other than the style guides?

Could you post the link?


 
Valeria Ranieri
Valeria Ranieri  Identity Verified
Local time: 15:03
French to Italian
+ ...
About phase 4 Apr 29, 2017

maroscz wrote:

First link to the tutorial page comes when you register for the test, before you get the SMS. Then there is the link on the Hermes homepage.

In my test I got 2 questions where there was a typo (wrong letter). I marked one example as having no errors. All the other examples clearly breached some of the timing rules stated in this document.

https://backlothelp.netflix.com/hc/en-us/articles/215758617-Timed-Text-Style-Guide-General-Requirements

I don't have Netflix subscription, but from what i read in recent weeks, the bad quality of the subtitles was the problem, that lead Netflix to change the way they handle the task, and made them come up with the Hermes platform.

Unfortunately it seems, that while they were trying to get rid of the subpar vendors, they chose a subpar vendor for the task.


I studied both the guides ("general requirements" and Italian), but that's only theory. I even learned by heart some of those rules but did not help me. In the videos I saw, there were no typos, so for all of them the problem must have been the timing. But in none of those videos the subtitles were clearly out of sync: I suppose there must have been very slight and subtle problems (if they arrived too early or too late, the difference between audio and text was so minimal my eyes couldn't detect it).

As for the quality of current subtitles in Netflix: how is one supposed to practice if the existing subtitles for their products are wrong? The week before taking the test I closely analysed their subtitles: they don't always correspond to their guidelines.

So, before taking the test we were provided with no video tutorials and couldn't even practice watching the content on Netflix because the text was often wrong. Moreover, it appears they're not taking into account the last 2 phases of actual translation...That's why I' m so disappointed.


 
José Henrique Lamensdorf
José Henrique Lamensdorf  Identity Verified
Brazil
Local time: 11:03
English to Portuguese
+ ...
In memoriam
A missing link in the loop Apr 29, 2017

maroscz wrote:

I don't have Netflix subscription, but from what i read in recent weeks, the bad quality of the subtitles was the problem, that lead Netflix to change the way they handle the task, and made them come up with the Hermes platform.

Unfortunately it seems, that while they were trying to get rid of the subpar vendors, they chose a subpar vendor for the task.


This is not specific to Netflix, but relates to the entire subtitling industry nowadays; perhaps the dubbing industry too.

The world has become global! Though this is some kind of oxymoron, it's true! A few days ago, using Kodi here in Brazil, I was able to watch what was purported to be live television from Kazakhstan! (I guess it was - couldn't understand a word of it.) Of course, I didn't expect that broadcast to have subtitles in Portuguese, however local transmissions of foreign movies/series in Brazil must have them.

I often see irate fellow translators complaining here, on Facebook, everywhere, about the bad quality and ridiculous mistakes they see in subtitles. Many say they wrote to the TV station, cable provider, etc. to no avail. My opinion is that they should be writing to SPONSORS instead. After all, a sponsor pays to have their commercials played in the, say, 8 intermissions throughout a specific show. If the spectator can't understand what they are watching on account of sloppy translation/subtitling/dubbing, they'll switch to another channel at the first of such intermissions, and the sponsor will have wasted 7/8 of their investment. As the French say, "Suivez l'argent!"

It is quite possible for cable TV stations, and much easier for both PPV and streaming TV companies (like Netflix) to devise some system to obtain, record, and analyze spectators' feedback on issues including the quality of translation AND (subtitling OR dubbing). As Hermes numbers are assigned to all translators, a computer system could automatically grade the providers on what really matters: customer satisfaction!

Though a "popular" translator could lure a crowd of friends to give him/her good grades, it is easy for such a system to check whether the voter watched the entire program, or merely hopped in just to vote, and thus validate or void such vote.

This would be much more effective than merely testing translators.

To illustrate, my late father worked for Philips before WWII. In the 1950s, I was a small boy, but I saw that he noticed lamps burning out at home every week. Of course, everybody tests bulbs at the store before buying them, and he was no exception; if it doesn't light up on the tester, take another one! In view of the circumstances, he began buying lamps from different brands and, every time he replaced a lamp, he'd write the date on the new one. In a few months, he had a table with brands and how long each lasted. Then he made a choice of buying only lamps from one or two brands, and I didn't see him replacing lamps so often any more.

All those bulb manufacturers probably had their QC in place, like Netflix does now. However this did not prevent some brands from being worse than others from the end-user's stance.

Hence, this is perfectly doable, and potentially effective.


 
maroscz
maroscz
Czech Republic
Local time: 15:03
English to Czech
Phase 4 Apr 29, 2017

Franjo Varšić wrote:

maroscz wrote:

First link to the tutorial page comes when you register for the test, before you get the SMS. Then there is the link on the Hermes homepage.

In my test I got 2 questions where there was a typo (wrong letter). I marked one example as having no errors. All the other examples clearly breached some of the timing rules stated in this document.

https://backlothelp.netflix.com/hc/en-us/articles/215758617-Timed-Text-Style-Guide-General-Requirements



When you say tutorial, you're referring to something other than the style guides?

Could you post the link?


No, I mean the style guides. It surprises me someone had a trouble with this section as those errors where clear and easy to spot in my test. For example the TTE ended several frames after the shot change, while the speech ended several frames before the change. Of course it could have been different in other language tests.


 
maroscz
maroscz
Czech Republic
Local time: 15:03
English to Czech
Hermes future Apr 29, 2017

José Henrique Lamensdorf wrote:

It is quite possible for cable TV stations, and much easier for both PPV and streaming TV companies (like Netflix) to devise some system to obtain, record, and analyze spectators' feedback on issues including the quality of translation AND (subtitling OR dubbing). As Hermes numbers are assigned to all translators, a computer system could automatically grade the providers on what really matters: customer satisfaction!

Though a "popular" translator could lure a crowd of friends to give him/her good grades, it is easy for such a system to check whether the voter watched the entire program, or merely hopped in just to vote, and thus validate or void such vote.

This would be much more effective than merely testing translators.

To illustrate, my late father worked for Philips before WWII. In the 1950s, I was a small boy, but I saw that he noticed lamps burning out at home every week. Of course, everybody tests bulbs at the store before buying them, and he was no exception; if it doesn't light up on the tester, take another one! In view of the circumstances, he began buying lamps from different brands and, every time he replaced a lamp, he'd write the date on the new one. In a few months, he had a table with brands and how long each lasted. Then he made a choice of buying only lamps from one or two brands, and I didn't see him replacing lamps so often any more.

All those bulb manufacturers probably had their QC in place, like Netflix does now. However this did not prevent some brands from being worse than others from the end-user's stance.

Hence, this is perfectly doable, and potentially effective.


Since I did the test, I have spared some time reading other parts of the Netflix's Backlot and their Techblog. With the picture I made about their approach, I'm absolutely positive that they are a company that will make the system work. There are many articles about gathering, storing and interpretting user data. Netflix even has their own departement specializing in artificial inteligence development. Not just for utilizing AI for their needs, but one involved in the basic research of the matter.

It was just the outsourced testing part I was reffering to in the previous post. I think we all agree it should have been done better. Now they have sofisticated system to analyze the big data, but they missed the chance to feed it quality data from the start.


 
bellaria
bellaria
Italy
Italian to English
+ ...
ITALIAN THRESHOLD Apr 29, 2017

ChachiArcola wrote:

I scored 81,33% and failed. So, the Italian threshold must be somewhere between my score and yours.


87,33% and failed, so the Italian threshold must be 88%.


 
Willian Meirinho
Willian Meirinho
Brazil
Local time: 11:03
Timing Apr 30, 2017

Willian Meirinho wrote:

Hi everyone!

I did the test twice and i'm a bit annoyed with the results.

First time i did it, it was everything swift, just to evaluate myself. One of the clips that i had to subtitle, i didn't finish and it wasn't in line with the standard of my language. Still, I've got a 74% score. This first try was arond 4 weeks ago and they took some 3 weeks to evaluate the test ang give me the results.

Second time. This one was 2 days ago. This time, i took time to study the rules of subtitling. I did all the multiple choice questions, even double checked with Google. Aside the part you need to spot the error on the subtitle cutscene, i'm pretty sure i nailed it 100%. At the subtitling part, this time i filled all the lines. I did everything according with Netflix requirements and double checked as well. Now the problem: My result appeared instantly after i submitted and it was just 78%. They didn't not even check the most important part of the test and NO WAY i got more than 10 questions wrong.

I was wondering if they take the time i took answering into consideration, because i double checked everything and submitted with little time left... I don't know. The whole thing seemed very unfair for me.

I'd like to know your opinion about it.

Congrats for everyone who passed.








Anyone else?


 
Yellow Panda (X)
Yellow Panda (X)
Slovakia
Local time: 15:03
Slovak language Apr 30, 2017

Hello.
I have just passed the test with 82.67 % in Slovak language. I would like to ask if someone has been contacted by Netlix and when will they start contacting people who pased.


 
Wojciech_ (X)
Wojciech_ (X)
Poland
Local time: 15:03
English to Polish
+ ...
No info Apr 30, 2017

Yellow Panda wrote:

Hello.
I have just passed the test with 82.67 % in Slovak language. I would like to ask if someone has been contacted by Netlix and when will they start contacting people who pased.


I don't think they will contact anyone soon as they first need to resolve the issues with the test. Secondly, it's not like they will suddenly start employing new translators in their thousands. I think this will be a gradual process. They will be in touch with their vendors., see how many of their translators have passed the test and then perhaps decide to hire some people. But I'm quite sceptical those who have passed will be inundated with Netflix projects.


[Edited at 2017-04-30 18:38 GMT]


 
Joanna Spychala
Joanna Spychala
Poland
Local time: 15:03
English to Polish
+ ...
After the test Apr 30, 2017

Passed, with 94.67% for English to Polish. The test was fun and pretty easy. I was afraid of ambiguous questions but fortunately, this was not the case (even though the differences between the answers were sometimes minor), with the exception of one question (section 3) where, in my opinion, two translations provided were correct and differed only very slightly in register/meaning. It was debatable which one was the better rendition of the original.

Generally, the test seems well-de
... See more
Passed, with 94.67% for English to Polish. The test was fun and pretty easy. I was afraid of ambiguous questions but fortunately, this was not the case (even though the differences between the answers were sometimes minor), with the exception of one question (section 3) where, in my opinion, two translations provided were correct and differed only very slightly in register/meaning. It was debatable which one was the better rendition of the original.

Generally, the test seems well-designed, checking a number of relevant skills (knowledge of idioms in both the source and target language, listening comprehension, accuracy, the ability to apply the relevant style guides). I guess they will manually evaluate the part with actual subtitling later, when/if they have some available work in the respective language, which is understandable, as such an evaluation requires lots of resources.

The time to complete the tasks could have been more limited, though. There was plenty of time to check some vocabulary and idioms in dictionaries/on-line, and to consult the style guides. On the other hand, this might be an accurate reflection of the actual job: you have some time to consult your sources, you just have to do it fast and you have to correctly identify the spots where it is actually needed.

I'm not a subtitler/audiovisual translator, though, so I may not know the specific requirements of this profession.
Collapse


 
isantillana
isantillana
Spain
Is there even a threshold? May 1, 2017

A friend of mine and myself took the test (both English into Spanish) within one day of each other. I failed at 86.67%, which stung a little as I've been subtitling for 15 years now. But that feeling turned into laughter when my friend passed... with 86%! Has anyone else seen something like this?

 
InBloom
InBloom
Italy
Local time: 15:03
English to Italian
Some questions May 1, 2017

Joanna Spychala wrote:

Passed, with 94.67% for English to Polish. The test was fun and pretty easy. I was afraid of ambiguous questions but fortunately, this was not the case (even though the differences between the answers were sometimes minor), with the exception of one question (section 3) where, in my opinion, two translations provided were correct and differed only very slightly in register/meaning. It was debatable which one was the better rendition of the original.

Generally, the test seems well-designed, checking a number of relevant skills (knowledge of idioms in both the source and target language, listening comprehension, accuracy, the ability to apply the relevant style guides). I guess they will manually evaluate the part with actual subtitling later, when/if they have some available work in the respective language, which is understandable, as such an evaluation requires lots of resources.

The time to complete the tasks could have been more limited, though. There was plenty of time to check some vocabulary and idioms in dictionaries/on-line, and to consult the style guides. On the other hand, this might be an accurate reflection of the actual job: you have some time to consult your sources, you just have to do it fast and you have to correctly identify the spots where it is actually needed.

I'm not a subtitler/audiovisual translator, though, so I may not know the specific requirements of this profession.



Hi Joanna,

So you translated the idioms with their corresponding idioms in your language for the phase 2? Or did you select the translation that fits better for the phrase but that's not actually the specular translation of that idiom in the corresponding idiom of your mother tongue?

[Edited at 2017-05-01 13:19 GMT]


 
brennie
brennie
Türkiye
English to Romanian
Possible May 1, 2017

isantillana wrote:

A friend of mine and myself took the test (both English into Spanish) within one day of each other. I failed at 86.67%, which stung a little as I've been subtitling for 15 years now. But that feeling turned into laughter when my friend passed... with 86%! Has anyone else seen something like this?



It happened to me as well. There's been talks according to which the threshold for my language (Romanian) was pretty high. I took the test and passed with 83%. Maybe thresholds vary from test to test. ,
My problem with this test was that the English captions were different than the spoken English from videos - which I find it weird, since I normally expected accurate captions. All in all, I found the test an useful experience.

I also read the lines where people had the intuition that only 4 parts of their test was assessed, as well as the ones who explained they submitted the video part in blank. I do not believe these stories, I think the test has general parameters of assessment for the video part too; it's hard to think that someone who is interested in working for Netflix would submit in blank, nor that he/she, in case they were not interested, would waste their time just to check how unprofessional the Hermes test is built.

[Edited at 2017-05-01 14:34 GMT]

[Edited at 2017-05-01 14:35 GMT]

[Edited at 2017-05-01 14:36 GMT]

[Edited at 2017-05-01 14:44 GMT]


 
María Marí Ros
María Marí Ros  Identity Verified
Spain
Italian to Spanish
+ ...
Me too May 1, 2017

isantillana wrote:

A friend of mine and myself took the test (both English into Spanish) within one day of each other. I failed at 86.67%, which stung a little as I've been subtitling for 15 years now. But that feeling turned into laughter when my friend passed... with 86%! Has anyone else seen something like this?


Yes, I failed (EN-SP) with 92%, which is quite weird.


 
Pages in topic:   < [1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27] >


To report site rules violations or get help, contact a site moderator:


You can also contact site staff by submitting a support request »

Netflix Hermes test







Trados Business Manager Lite
Create customer quotes and invoices from within Trados Studio

Trados Business Manager Lite helps to simplify and speed up some of the daily tasks, such as invoicing and reporting, associated with running your freelance translation business.

More info »
Trados Studio 2022 Freelance
The leading translation software used by over 270,000 translators.

Designed with your feedback in mind, Trados Studio 2022 delivers an unrivalled, powerful desktop and cloud solution, empowering you to work in the most efficient and cost-effective way.

More info »