Tweaking Bayes and how Chance favors the prepared

Published Sun, May 23 2004 8:08 | William

It was a few years ago in grad school that I was first introduced to many advanced statistical concepts.  Back then there was  a good amount of software that could perform high level statistical analysis like Multi-Linear Regression, Monte Carlo simulations and many more. My professors noted that software was only to get more powerful and easier to use, so the real focus of learning these techniques was in interpretation.  I know a lot of people don't understand this idea and isist that being able to do regressions by hand for instance, is the metric that measures that someone really knows how to do it.  I think they both have value and I never heard any of my professors argue otherwise.  But their point was that 70 years ago, the whole focus should have been on doing the regression correctly.  To that end, it was the only issue b/c if you didn't do it correctly, all of the analytic understanding in the world wouldn't provide anything worthwhile.  Due to the complexity of some of these techniques, not having a calculator would certianly make the whole thing more error prone.  Well, I still remember how to interpret multiple linear regressions for instance and most everything else I learned.  I couldn't do one by hand unaided by a text boox or formula sheet to save my life (at least not without studying it again.  I can however still fire up Excel, use the Analysis tools and start regressing.   I can still check spot low correlational values and autocorrelation from a mile away.  I still remember the adjusted R Square caculation and why it matters.  But this is of little value without being able to correctly identify what the results mean.  I'm trying to slice and dice some numbers a few different ways though and as a prelim, here's what I'm finding out.

1-  I'm much more likely to become wealthy from winning an international internet lottery or doing a favor for someone in Sub-Sahara Africa than I will from software development

2- I've positioned myself in life in such ways that I've indirectly developed friendships with more deposed dictators, their wives, their friends, their assistants than most other people.  I've also statisitically significantly developed friendships  indirectly  that are so deep that people are willing to trust me with large sums of money even though I don't know them.  I have yet to have an American, let alone a family member, offer to pay me a few million dollars to help them with anything

3- Considering the racial tension that's commonplace in Zimbabwe, South Africa  and Liberia, I've somehow transcended the fact that I'm of Eurpean descent, currently live in the US and happen to be very light skinned (I have less of a tan than Casper the Ghost).  This is quite a feat.  For instance, people with my exact profile as mentioned above, have their land Confiscated and face Horrific  consequences if they put up any resistance.  The fact that my ethnic background isn't much different from those of the people that are often tortured, does not seem to be a noteworthy predictor variable in whether or not Mugabe or any of this friends, let alone the Taylor family, will trust me with extremely large sums of money and pay me an inordinate commission for even a little bit of help.

One can use the day of the week as a predictor variable as to when i'll be contacted by a deposed dictator's  wife or associate.  Sunday and Tuesday seem to be the surest bets that I'll hear from them.  Moreoever, having been contacted by them previously seems to have no deterrent effect whatsoever on whether or not they'll conact me again.

I'm over 3000 times more likely not to be addressed by my name in an introduction from deposed dictators and their wives.  There is 0 probability that a story, anecdote or reference to something we did together before will be brought up if a famous dictator or dictator's wife is offering me money.

The strongest predictor that an email will be from a dictator, former dictator or their friend or family is the existence of the words “Dear“ coupled with either “Sir“ or “Madam“ in the introduction.

4-  Being well educated and having tremendous wealth is not a predictor variable indicating that people know how to use Word's spell check feature, nor does it mean that one is capable of composing coherent sentences

5-  I'm roughly 4.72 million times more likely to be contaced by a dictator's wife than the dictator himself.  If we look at deposed dictators, the number shoots up drastically to almost 80 million

6- There is an epidemic of small sexual features in American men as well as an epidemic of impotence

7- It's almost 11,128 times easier to obtion 'l3gal' pharmaceuticals without a d0ct0rs pr3sce1ption via the internet than it is to legally procure such a thing.  -  This is based on the assumption that doctors will not automatically prescribe presecription drugs without an indication of need. Another analysis would be comparing how many 'bricks and mortar' pharmacies advertise narcotics and prescription drug availability without a prescription compared to the number of internet pharmacies that do

There are so many other patterns and probabilities that I could track, I just don't have the time.  If you can think of anything you'd like me to analyze, please let me know.  I'll need to know a few things, specifically about the email compared to other ones, but I'll be glad to do it.  The whole thing is really about Bayesian Filtering and using more advanced techniques to battle spam.  Bayesian filtering is a very smart idea and it's quite effective, certainly much more so than generic block lists which pretty much guarantee that they'll block out important emails while letting spam through.  But with all of the internet scams out there, it may be of benefit to post profiles of a scam and have ISPs' and everyone else start pushing people to read them.  The same is done with Viruses and attachments and although there are still some folks out there that don't get it, a lot of good has been done by it. Maybe it's time the same is done with Scams.  If people didn't fall for this crap, Spammers will have to spend more money to reach even more people, as this would decrease the success rate of spamming. Every one of them that goes out of business is music to my ears.  So maybe the solution all resides in math?


PREMIUM TRUST AGENCY
CUSTOMER SERVICE
510N
SACRAMENTO, CA85914
FAX:001-270-717-2089
(ACCREDITED LICENSED AGENT TO GLOBAL LOTTERY INTERNATIONAL)
AN AFFILIATE OF GLOBAL TRUST AGENCY
Daalwerk 100c p/c 1102 RL Amsterdam zuidoost
AMSTERDAM, THE NETHERLANDS
TEL:+31-6424-985-30 FAX: +31-206-006-491
globst012@netscape.net
Ref. Number: 992/477/4067
Batch Number: 006371721-DC08
CONGRATULATIONS!!!
Dear winner,
We are pleased to inform you of the result of
the Lottery Winners International programs held on the 26th of november,
2003. Your e-mail address attached to ticket number 57517465896-8580
with
serial number 0907-635 drew lucky numbers 7-13-88-32-41-11 which
consequently
won in the 1st category, you have therefore been approved for a lump
sum
pay out of US$ 850,000.00 (Eight hundred and fifty thousand United States Dollars)
CONGRATULATIONS ones more!!!
Due to mix up of some numbers and names, we ask that you keep your
winning information confidential until your claims has been processed
and
your money Remitted to you. This is part of our security protocol
to avoid double claiming and unwarranted abuse of this program by some
participants.
All participants were selected through a computer ballot system drawn
from over 20,000 company and 30,000,000 individual email addresses and
names
from all over the world. This promotional program takes place every
three
year.
This lottery was promoted by the software corporation to compensate
some few
individuals with website and email addresses,  we hope with part of
your
winning you will take
part in our next year USD20 million international lottery.
Pease note that your lucky winning numbers falls within our european
booklet
representative office in europe as as indicated in your play couponTo
file
in for your claims please contact your ,
FIDUCIAL Agent MRS:SARAH HELMOT.
TEL: 31-642-498-530
FAX: 0031-206-006-491
Email:sarahelmot@netscape.net
Remember, all winning must be claimed not later than{ 30th of JUNE
2004}
and  winning claims begins {1ST JUNE 2004}. After this date all
unclaimed funds will be included in the next stake.
Please note in order to avoid unnecessary delays and complications
please remember to quote your reference number and batch numbers in all
correspondence. Furthermore, should there be any change of address do
inform our agent as soon as possible.
Congratulations once more from our members of staff and thank you for
being part of our promotional program.
Note: Anybody under the age of 18 is automatically disqualified.
Sincerely yours,
Mrs.DORIS GROTE.
Lottery Coordinator




                                                                     


 

Comments

# William said on May 28, 2004 7:42 PM:

Yes, it is truly statistically amazing just how many people are continually asked for help with transferring large sums of money on a daily basis by some person from South Africa (or wherever), how many offers claiming to increase the size of body parts are sent each day, and very curiously, why is it that I receive nearly daily announcements that my mortgage loan has been approved, when I don't even own a house? Hmmm...

# William said on May 29, 2004 3:16 AM:

I agree, I'm all about having the bi663st P3ni5 that I can have, but If I have to increase my breast size too, well, just not sure I can live with that.

Search

This Blog

Tags

Community

Archives

News

My other sites

Cool Stuff

Book Stuff

Security

ORM

Data Access

Funny Stuff

Compact Framework Stuff

Web Casts

My KnowledgeBase Articles

My MVP Profile

Design Patterns

Performance

Debugging

Remoting

My Fellow Authors

My Books

LINQ

Misc

Speech

Syndication

Email Notifications