Being recognized by one’s peers can be a happy moment. It’s far better than being ignored. But what happens when your name appears on a winner’s list and you don’t know why it got there?
What about awards lists that name people you know don’t deserve to be there or there are exceptional people who were left off the list?
I wrote about a contest in December that didn’t make sense to me. With today’s winners announcement for that contest came a viable rant by Rae Hoffman, CEO of Outspoken Media. As she wrote in Most Influential Online Marketers of 2009 FAIL for her company’s blog,
In addition to the obvious haste in publishing the final results (at the time of this writing there are numerous typos in the bios of multiple marketers), Invesp decided to taint both the nomination and selection process, left out some of the most obvious and influential marketers in the industry, while including some folks that you’d have a hard time even having heard of, much less name anything influential they did for the community at large in 2009.
A lot of the marketer bios sound as if they were scraped from conference bio pages and it was obvious no real research was done into deciding or explaining why some of the marketers on the list were considered influential enough to make the list.
With the SEMMYS coming out, as well as numerous other yearly nods, I wish someone would do them with some common sense, without bias by judges (because not everyone in the SEO/M industry likes or gets along with each other), and tell us what the winning criteria are.
This past year I feel I wrote some of my best articles ever on the topics of search engine optimization and usability. I wrote them for 4 other publications, however. Unless THEY are followed, I most likely will not get any notice for that work. It is something that contests don’t take into consideration. There are no categories for cross-skilled people and no categories for experts who conduct training or live on the speaker circuit. It’s as if those contributions to their field don’t matter in the least and I take issue with that.
A contest worth its salt to me would:
1. Obtain extensive user generated feedback. It should be mandatory to give a solid reason for making a nomination. Being a fan should not count. Related: Judges should be experts in the category they are judging.
2. Allow have user instructions that make it easy to understand how the process works.
3. Put up a form for feedback on that process and correct the issues that come in.
4. Explain the judging criteria. What makes someone a winner? Is it something they did?
5. Define your terms. What does Internet Marketer mean, exactly? Does it include all the branches of online marketing and if so, isn’t it logical to break out into categories?
6. Research nominees and candidates, please. Some people are very clever at hiding behind their computer monitor, making it look like they even have a business when in fact, they do not. Check employment history. Check to see if they left the industry. Just because someone was once well known doesn’t mean they are still active anymore.
7. Get references for any work related accomplishments. Make it known exactly what a winner’s specific achievement is.
8. Give examples of what you are looking for in a winner rather than accepting anything.
9. Do not cut and paste bios from conference materials or their site bios. If they won, allow winners the opportunity to tell you what they want published. Related: Avoid grammatical and spelling errors. It just makes your contest look sloppy.
10. Do not use the contest to market yourself. This is link bait, not a true competition.
Maybe it’s part of the gene pool of marketers to keep rewarding themselves. I crack up when I speak at conferences because I find that in the real world, nobody knows me. I’ve been in the SEO and Usability fields for going on 15 years and yet when I was hired to train new SEO’s for a company in November, none of the trainees had ever heard of me. So, if “influencing the industry” is a criteria for winning, who the hell cares?
I like to be acknowledged, don’t get me wrong. I love a good pat on the head once in awhile. But I want to earn it. I want to know what I did that you liked. By the same token, it is sad when I know I’ve accomplished something that goes unnoticed. I think this is also what bothers other people. We’re each unique. I know of no one who combines SEO, software QA testing, usability testing and information architecture like I do. I know some people who may have two of these skills and that’s all that is required of them or all they are interested in doing. I’m usually excluded from contests because I don’t fit into their square peg.
Finally, there are some people in the internet marketing field who fit in somewhere as part of a team, with their specialty, such as copy writing. Categories most often missing in Internet marketing contests include mobile marketing, video marketing, online radio, social media, teaching and education (forums, schools), niche blogs and search usability (findability).
So my final request for those who would run awards and contests is to be sure you know and fully understand your own industry first.
Do you enjoy discussions on web design and marketing? Membership is free at Cre8asiteforums.com.
- Follow me on Twitter to communicate and stay connected.
80 Website Usability Heuristics; only $5, PDF
80 Website Conversions Heuristics; only $5, PDF
{ 9 comments… read them below or add one }
Kim, I appreciate your passion on this topic. But I’m curious, at least where the SEMMYS are concerned (since that’s the only awards/competition I know all the details of) … how would you propose that judges be removed from the process? It seems physically impossible, in my mind at least, for some system to be developed that would automate the entire process. Plus, since the SEMMYS don’t make a dime (and actually cost me a TON of time and a wee bit of money), how would I afford such a system?
Matt, for the SEMMYS, which btw are far more well organized and handled with professionalism, my only beef is that the judges for categories should have a proven, demonstrated level expertise for the categories they are assigned to judge. I’m not sure how to prevent personal feelings from getting in the way. There are camps of people in the SEO/M.SMM industry who don’t get along, or don’t like one another and in some cases really object to one another. I know that personally I can not be treated objectively by my peers, so I don’t expect to be judged fairly by them either. A jury pool of peers may not be the way go, but Matt, I have no suggestions for another way, since the SEMMYS are designed as a peer to peer contest.
I would also like to see the reasons why something is chosen. Are there certain criteria for example? What makes the winner a winning piece?
I am a judge for Andy Beal’s annual SEO contest. I’m not sure if he sends our feedback to contestants but I always try to backup my choices in the event that he might. Losing entries deserve to understand how they might improve, in my opinion.
Trust me when I say I know deeply what it is like to found and run something made up of volunteers that is funded out of my own pocket or with donations. I’ve run the same type of ship since 1998 (Cre8asiteforums). My moderators often have better ideas for things than I do.
Matt, one more thing…if I understand correctly, the SEMMYS look at blogs/bloggers/articles on topics. Are there provisions for bloggers/article writers whose articles appear on publications that are not their own? For ex., many people write for SEL. They own the copyright to those articles. Are blog posts and articles written for outside sources counted towards an authors contributions? Do magazines who have a blog count or is it just blogs?
I would ask for feedback on interest in podcasts too. Is there an interest in a section for accessibility and internet readers who listen to articles? Talk about a contribution to a specific market! If there is such a thing, I would love to support it.
Kim your absolutely right in that it is difficult to lump all the various online marketing specialties into a single category (we can attest to that!). Maybe next year we’ll have an overall ranking, and a ranking by specific online marketing category (SEO, usability, SMO, etc.) Thanks for your feedback, it is very valuable to us!
Thanks for your thoughtful replies, Kim. I’ll respond in order to some of your concerns and suggestions.
1) I always try to have judges be as smart as possible about the topic they’re judging. This year, for example, Chris Churchill and Adam Audette are judging the PPC category — both very smart in that area (not to mention several other areas). But I also have a rule that no one can judge category in which s/he is nominated, nor can they judge a category in which his/her site is nominated, nor in which s/he has direct ties. This makes it incredibly difficult to assign judges. A couple examples:
Rand/SEOmoz: There are at least a half-dozen of his staff members who write on their blog, not to mention all the user-generated content — some of which gets nominated, too. While I trust Rand implicitly, it wouldn’t look right for him to be judging a category in which SEOmoz is nominated. So this rule automatically precludes him from judging from about half the categories.
Lisa Barone: Lisa writes for Outspoken Media’s blog, for Small Business Trends, and has occasionally written for Search Engine Land and other sites during 2009. She’s nominated in a LOT of categories. Outspoken Media and Small Business Trends are also nominated in a lot of categories. I trust Lisa implicitly, too, but she shouldn’t be judging categories where the sites she’s affiliated with are among the nominees.
I could go on with examples like Barry Schwartz, who can’t judge any category where Search Engine Land is nominated. Aaron Wall would be a great SEO category judge, but his blog is nominated several times in that category (and several others). And making judging assignments even tougher is the amount of guest blogging that goes on. The most difficult part of my job is assigning the judges. I have charts and spreadsheets and handwritten notes with scratches and scribbles all over them — you name it. If I had longer hair, I’d be tearing it out every year I do this.
When choosing judges, I’ve tried to choose the smartest people possible — people who aren’t one-trick ponies. So even though they may not end up judging the category they’re most associated with, they’re still smart enough to know good content when they see it. I’m not a PPC guy by any means, but I think I could judge what a good PPC article looks like. (I’m not a judge, for the record.) You even said in your comment that you judge an SEO contest, yet I believe you consider yourself first and foremost a Usability person, right? My point is that, for the most part, many of the categories run together and are closely related. It shouldn’t be tough for a person who’s considered an expert in Category A to also be smart enough to judge Category B. And in the rare case when a judge tell me s/he doesn’t feel qualified to judge a certain category, I don’t assign him/her there.
2) Personal feelings & peer judging. Yep, we all have feelings. But we’re all professionals, too. I’ve nominated a couple dozen articles this year, and several of them are from people who’ve hurt/attacked me in the past in various forums — including about the SEMMYS! You assume that emotions will always get in the way of fairness, but putting emotion aside isn’t tough for everyone. I’ve told the nominating committee and judges several times that they have to be professional and fair. I believe they are. If I’m aware of particularly sensitive personal issues, I’ll make sure certain people don’t judge certain categories. And judges know they can tell me if they’re uncomfortable with the assignment I give them. So again, I’ll ask: If peer judging is not a good idea, how do I automate this and who pays the costs?
3) Why is something chosen? We have a page on the site that explains how the process works:
http://www.semmys.org/how-it-works/
There are 8 people who nominate articles throughout the year. They do so with very general instructions from me: When they find great content related to online marketing, nominate it. Doesn’t matter if it’s an article, blog post, podcast, video, whatever. Doesn’t matter who wrote it, where it was published, etc. Only matters that it was written during the calendar year and is great. Then we post all the nominated content. I assign 20+ judges, usually 2-3 per category, to read the nominees and — as a group — come to a consensus on a smaller group of finalists. (So, for example, Chris and Adam will have to agree on the PPC finalists this year.) We post the list of finalists, and then public voting decides who the winners are in each category. This last step isn’t perfect, but it does eliminate any of the personal feelings stuff you mentioned. And even though I don’t always agree with who the voters choose as a winner, I’d rather have them do it than have a small group of industry insiders do it. Plus, public voting puts the burden on the finalists to prove their worth to a degree by making them market their content if they want to win. I’m okay with that.
I’m sorry to say that, if you’re looking for explanations for why each nominated article was nminated, that’s just not feasible with 100-200 articles nominated each year. And it’s not possible to
4) On your second comment — I think if you look through past nominees and winners, you’ll see that there are no rules limiting a nominee to a certain publication. Pretty sure you’re nominated this year for something not on this blog. Lisa B. is nominated for her articles on several sites.
Looks like some of my comment went MIA. Sorry about that. At the end of #3, was trying to say that it’s also not possible to get Joe Q. Public Voter to explain every vote s/he makes for the winners.
Matt, you’re thanking me? Crazy talk. You took a good chunk of time to answer my questions. (Is their an award for that?
I ended up with more questions and a deeper appreciation for you and the SEMMYS.
Peer to peer review may work in some fields but I have doubts about marketers. They are competitive by nature.
“,,,no rules limiting a nominee to a certain publication”….good. So it could be author, not by pub. Thank you.
“I’m sorry to say that, if you’re looking for explanations for why each nominated article was nominated, that’s just not feasible with 100-200 articles nominated each year”…Interesting. My perspective is that I am viewing this as a teaching tool, in which feedback fits in.
“Only matters that it was written during the calendar year and is great…” Define “great”, LOL The rub, for me, is that great means ??? I was contacted by someone who read an article of mine who said I had just written his thesis paper for him. Quite a compliment, but was it great for others?
The chance to narrow down options by readers is something I always liked. Would that be a time to get feedback for the nominees?
You are more optimistic than I. I listen to a lot of people and know how I am treated as well at conferences by a few folks. I know what its like to be on the receiving end of a hate campaign. I have seen and experienced a lack of professionalism by peers and so hence, I am grumpy about the topic.
costs…I face the thing with the forums. Ads drive the community nuts so I try to avoid them. We get donations but not unless I periodically admit we need it to pay for domains, server and software. People forget or expect. I can’t remember if you ask for help….bad kimmy.
Here locally, there is an All Star thing they do for high school football, where coaches from other teams scope out talent and nominate boys from competitor teams for awards. They are not allowed to nominate their own talent. It bugs them when someone they want in the All Star rank isn’t selected but I guess it is a way of being objective.
You have done your best to answer my questions without freaking out on me. Thank goodness! With online communication, it is hard to know the feelings behind words. I am your friend. I support the SEMMYS, even more so since you strive so hard to make it the best. Thank you sincerely for responding.
Thx again, Kim, for the kind words.
The SEMMYS aren’t a teaching tool. They’re for our industry. The OSCARS don’t exist to teach filmmaking, right? You don’t see the Academy up there giving feedback to the directors who don’t win about what they could’ve done better. Maybe this is part of the issue here — your impression of it is different from what the SEMMYS are.
Define “great” … aye, that’s the beauty of it. You can’t define great. It’s different for everyone, and that guarantees that we get a nice variety of nominated posts every year. Again on the OSCARS theme — define “Best Picture.” They don’t. The GRAMMYS don’t define “Best New Artist.” It might be sales, it might be impact, buzz, whatever. I don’t agree that all the nominees are “great,” but I respect and value the opinion of the committee and if they think something’s great, so be it. If they nominate something that’s not great, either the judges or the public will usually make sure it doesn’t get an award.
“….I have seen and experienced a lack of professionalism by peers and so hence, I am grumpy about the topic.” — at the risk of getting all Dr. Phil here, why be grumpy about it? It’s out of your control. Only thing you control is your reaction to it. People that act that way — I don’t let them bother me. They win, they have power over me when that happens. I ignore them as much as possible and surround myself with better people.
Anyhoo, fire away with questions anytime and I’ll do my best to answer.
Perhaps it is in fact my own expectation. I always viewed lists/awards as something to aspire to. The winners have done “something” right, and I want to learn from that.
“They win, they have power over me when that happens. I ignore them as much as possible and surround myself with better people” If I only surrounded myself with better people, I would never learn how to get along with everyone.
I was hoping other readers would see an opportunity to ask you questions too but maybe they are all smarter than I am
{ 1 trackback }