Классификация методом Naïve Bayes.
Этот алгоритм применяется сразу к исходным данным без какого либо их изменения. Т.е. не приходилось использовать фильтры. Пример вывода результата работы алгоритма в окне Weka представлен в листинге 2.
Листинг 2
Naive Bayes Classifier
Class >50K: Prior probability = 0.22
age: Normal Distribution. Mean = 43.4644 StandardDev = 11.4248 WeightSum = 87 Precision = 1.2166666666666666
workclass: Discrete Estimator. Counts = 58 9 8 2 7 6 1 1 (Total = 92)
fnlwgt: Normal Distribution. Mean = 198709.5181 StandardDev = 133275.5372 WeightSum = 87 Precision = 2374.035714285714
education: Discrete Estimator. Counts = 31 13 1 14 6 5 6 1 1 1 16 1 2 3 1 1 (Total = 103)
education-num: Normal Distribution. Mean = 12.1034 StandardDev = 2.068 WeightSum = 87 Precision = 1.0
marital-status: Discrete Estimator. Counts = 78 4 8 1 1 1 1 (Total = 94)
occupation: Discrete Estimator. Counts = 4 11 1 12 15 27 3 3 6 5 5 1 4 1 (Total = 98)
relationship: Discrete Estimator. Counts = 10 3 69 8 1 2 (Total = 93)
race: Discrete Estimator. Counts = 78 7 2 1 4 (Total = 92)
sex: Discrete Estimator. Counts = 13 76 (Total = 89)
capital-gain: Normal Distribution. Mean = 3048.4453 StandardDev = 11362.802 WeightSum = 87 Precision = 4347.782608695652
capital-loss: Normal Distribution. Mean = 254.023 StandardDev = 671.2905 WeightSum = 87 Precision = 260.0
hours-per-week: Normal Distribution. Mean = 46.7126 StandardDev = 13.106 WeightSum = 87 Precision = 2.0
native-country: Discrete Estimator. Counts = 78 2 1 1 2 2 1 3 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 1 1 1 2 1 (Total = 127)
Class <=50K: Prior probability = 0.78
age: Normal Distribution. Mean = 38.0743 StandardDev = 14.3222 WeightSum = 313 Precision = 1.2166666666666666
workclass: Discrete Estimator. Counts = 213 28 9 14 21 13 1 1 (Total = 300)
fnlwgt: Normal Distribution. Mean = 186365.596 StandardDev = 98559.3938 WeightSum = 313 Precision = 2374.035714285714
education: Discrete Estimator. Counts = 37 73 17 118 2 9 17 9 7 9 6 2 11 4 6 2 (Total = 329)
education-num: Normal Distribution. Mean = 9.4696 StandardDev = 2.3686 WeightSum = 313 Precision = 1.0
marital-status: Discrete Estimator. Counts = 122 49 113 15 12 8 1 (Total = 320)
occupation: Discrete Estimator. Counts = 8 47 40 22 21 36 15 25 42 20 18 2 9 1 (Total = 306)
relationship: Discrete Estimator. Counts = 11 50 106 91 16 45 (Total = 319)
race: Discrete Estimator. Counts = 262 10 4 6 36 (Total = 318)
sex: Discrete Estimator. Counts = 117 198 (Total = 315)
capital-gain: Normal Distribution. Mean = 180.5788 StandardDev = 867.4726 WeightSum = 313 Precision = 4347.782608695652
capital-loss: Normal Distribution. Mean = 66.4537 StandardDev = 383.1005 WeightSum = 313 Precision = 260.0
hours-per-week: Normal Distribution. Mean = 39.4569 StandardDev = 12.5934 WeightSum = 313 Precision = 2.0
native-country: Discrete Estimator. Counts = 280 1 1 2 2 2 1 2 1 1 1 3 1 1 1 3 2 1 3 1 8 1 3 1 3 1 1 1 2 1 1 2 1 1 2 1 1 1 1 1 1 (Total = 345)
Time taken to build model: 0.08 seconds
=== Stratified cross-validation ===
=== Summary ===
Correctly Classified Instances 336 84 %
Incorrectly Classified Instances 64 16 %
Kappa statistic 0.5179
Mean absolute error 0.165
Root mean squared error 0.3563
Relative absolute error 48.3537 %
Root relative squared error 86.3527 %
Total Number of Instances 400
=== Detailed Accuracy By Class ===
TP Rate FP Rate Precision Recall F-Measure Class
0.598 0.093 0.642 0.598 0.619 >50K
0.907 0.402 0.89 0.907 0.899 <=50K
=== Confusion Matrix ===
a b <-- classified as
52 35 | a = >50K
29 284 | b = <=50K
Этот метод основывается на анализе данных по всем исходным переменным. Формулируются правила, в условных частях которых сравниваются все независимые переменные с соответствующими возможными значениями. Оценив ошибки, выбирается переменная, для которой ошибка набора минимальна. Для каждого значения атрибута обучающего множества приводится его вероятность, а также выкладки по всем атрибутам тестового множества, включающие в себя средние значения и точность (количество совпадений тестового набора и обучающего множества) для численных (перечислимых) типов данных. В результате кросс-проверки получаем достаточно высокий процент верной классификации (84%), а также относительно небольшую среднюю абсолютную ошибку.
В результате после анализа данных на основе Classifier model можно сделать выводы о людях имеющих заработок больше или меньше 50К. К примеру в нашем случае можно сделать вывод, что те кто зарабатывает >50K работают в среднем на 7 часов в неделю больше чем те, кто зарабатывает <50K (hours-per-week: Normal Distribution. Mean = 46.7126 и hours-per-week: Normal Distribution. Mean = 39.4569 соответственно). Т.е. для числовых значений показывается среднее значение для данного класса, а также отклонение и точность. Для значений номинального типа показывается сколько раз встречается какое-то номинальное значение в конкретном классе. Например при заработке >50K 58 человек имеют workclass private (workclass: Discrete Estimator. Counts = 58 9 8 2 7 6 1 1 (Total = 92)) тут перечислены количества значений, перечисленных в тех данных, которые мы добавляли для получения arff «@ATTRIBUTE workclass {Private, Self-emp-not-inc, Self-emp-inc, Federal-gov, Local-gov, State-gov, Without-pay, Never-worked»})
В итоге получим, что человек с доходом >50К (их 22%) должен удовлетворять следующим условиям:
возраст: 43.4644+-11.4248
рабочий класс: 58 9 (Private, Self-emp-not-inc)
fnlwgt: 198709.5181+-133275.5372
образование: 31 13 14 16 (Bachelors, Some-college, HS-grad, Masters)
степень образования: 12.1034+-2.068
статус в браке: 78 (Married-civ-spouse)
занятость: 11 12 15 27 (Craft-repair, Sales, Exec-managerial, Prof-specialty)
семейный статус: 10 69 (Wife, Husband)
раса: 78 (White)
пол: 13 76 (Female, Male)
капитальная прибыль: 3048.4453+-11362.802
капитальный убыток: 254.023+-671.2905
рабочих часов в неделю: 46.7126+-13.106
родная страна: 78 (United-States)
А человек с доходом <=50К (их 78%) должен удовлетворять следующим условиям:
возраст: 38.0743+-14.3222
рабочий класс: 213 28 (Private, Self-emp-not-inc)
fnlwgt: 186365.596+- 98559.3938
образование: 37 73 118 (Bachelors, Some-college, HS-grad)
степень образования: 9.4696+- 2.3686
статус в браке: 122 49 113 (Married-civ-spouse, Divorced, Never-married)
занятость: Counts = 47 40 36 42 (Craft-repair, Other-service, Prof-specialty, Adm-clerical)
семейный статус: 50 106 91 45 (Own-child, Husband, Not-in-family, Unmarried)
раса: 262 36 (White, Black)
пол: 117 198 (Female, Male)
капитальная прибыль: 180.5788+- 867.4726
капитальный убыток: 66.4537+- 383.1005
рабочих часов в неделю: 39.4569+- 12.5934
родная страна: 280 (United-States)
Классификация методом ID3.
Этот алгоритм требует чтобы во входном наборе содержались только данные номинального типа. Для этого выбираем filter->unsupervised->attribute->RemoveType и удаляем все типы данных кроме nominal. Применение фильтра RemoveType изображено на рисунке 2.
Рисунок 2
После применения фильтров в наборе остаются только данные номинального типа. С ними и продолжает работу алгоритм. Далее для работы алгоритма необходимо отсутствие пустых значений. Чтобы осуществить это применяется фильтр ReplaceMissingValues, заменяющий пустые значения средними. Пример вывода программы изображен в листинге 3.
В нашем случае разбиение производится по переменной Income и дерево получается очень сильно разветвленным и не очень точным. Также возрастает средняя ошибка дерева, и при этом 15% из исходных выкладок не классифицируются.
Листинг 3
education = Bachelors
| relationship = Wife
| | workclass = Private: >50K
| | workclass = Local-gov
| | | native-country = United-States: <=50K
| | | native-country = Germany: >50K
| | workclass = State-gov: >50K
| relationship = Own-child
| | workclass = Private
| | | occupation = Exec-managerial: >50K
| | | occupation = Prof-specialty: <=50K
| | workclass = Self-emp-not-inc: <=50K
| | workclass = Local-gov: >50K
| | workclass = State-gov: <=50K
| relationship = Husband
| | occupation = Craft-repair: >50K
| | occupation = Sales
| | | workclass = Private: >50K
| | | workclass = Self-emp-not-inc: <=50K
| | | workclass = Self-emp-inc: >50K
| | occupation = Exec-managerial: >50K
| | occupation = Prof-specialty
| | | workclass = Private
| | | | native-country = United-States: >50K
| | | | native-country = Italy: <=50K
| | | workclass = Self-emp-not-inc: <=50K
| | | workclass = Federal-gov: >50K
| | | workclass = State-gov: <=50K
| | occupation = Handlers-cleaners: >50K
| | occupation = Adm-clerical: <=50K
| | occupation = Farming-fishing: >50K
| | occupation = Transport-moving: <=50K
| | occupation = Protective-serv: >50K
| relationship = Not-in-family
| | occupation = Tech-support: <=50K
| | occupation = Craft-repair: <=50K
| | occupation = Other-service: <=50K
| | occupation = Sales: >50K
| | occupation = Exec-managerial: <=50K
| | occupation = Prof-specialty: <=50K
| | occupation = Machine-op-inspct: <=50K
| | occupation = Adm-clerical: >50K
| | occupation = Transport-moving: <=50K
| relationship = Other-relative: <=50K
| relationship = Unmarried: <=50K
education = Some-college
| marital-status = Married-civ-spouse
| | occupation = Tech-support: >50K
| | occupation = Craft-repair
| | | workclass = Private: <=50K
| | | workclass = Federal-gov: <=50K
| | occupation = Other-service: <=50K
| | occupation = Exec-managerial
| | | workclass = Private
| | | | race = White: <=50K
| | | | race = Black: <=50K
| | | workclass = Self-emp-inc: <=50K
| | occupation = Prof-specialty
| | | race = White: <=50K
| | | race = Amer-Indian-Eskimo: >50K
| | | race = Black
| | | | workclass = Private: <=50K
| | occupation = Handlers-cleaners: <=50K
| | occupation = Machine-op-inspct: >50K
| | occupation = Adm-clerical: <=50K
| | occupation = Farming-fishing: <=50K
| | occupation = Transport-moving
| | | workclass = Self-emp-not-inc: >50K
| | | workclass = Self-emp-inc: <=50K
| | occupation = Protective-serv: <=50K
| marital-status = Divorced: <=50K
| marital-status = Never-married: <=50K
| marital-status = Separated: <=50K
| marital-status = Widowed: <=50K
| marital-status = Married-spouse-absent: <=50K
education = 11th: <=50K
education = HS-grad
| occupation = Tech-support: <=50K
| occupation = Craft-repair
| | relationship = Own-child: <=50K
| | relationship = Husband
| | | workclass = Private
| | | | race = White
| | | | | native-country = United-States: <=50K
| | | | | native-country = Germany: <=50K
| | | | race = Amer-Indian-Eskimo: <=50K
| | | workclass = Self-emp-not-inc: <=50K
| | | workclass = Self-emp-inc: <=50K
| | | workclass = Local-gov: <=50K
| | relationship = Not-in-family
| | | workclass = Private: <=50K
| | | workclass = Self-emp-not-inc: >50K
| | relationship = Other-relative: <=50K
| | relationship = Unmarried: <=50K
| occupation = Other-service: <=50K
| occupation = Sales
| | marital-status = Married-civ-spouse
| | | workclass = Private: <=50K
| | | workclass = Self-emp-inc: >50K
| | marital-status = Never-married: <=50K
| | marital-status = Separated: <=50K
| | marital-status = Widowed: <=50K
| occupation = Exec-managerial
| | relationship = Wife: >50K
| | relationship = Own-child: <=50K
| | relationship = Husband: <=50K
| | relationship = Not-in-family: <=50K
| occupation = Prof-specialty
| | relationship = Wife: >50K
| | relationship = Own-child: <=50K
| | relationship = Husband: <=50K
| | relationship = Not-in-family: <=50K
| | relationship = Other-relative: <=50K
| occupation = Handlers-cleaners
| | workclass = Private: <=50K
| | workclass = Self-emp-not-inc: >50K
| occupation = Machine-op-inspct: <=50K
| occupation = Adm-clerical: <=50K
| occupation = Farming-fishing
| | workclass = Private: <=50K
| | workclass = Self-emp-not-inc
| | | marital-status = Married-civ-spouse: >50K
| | | marital-status = Married-spouse-absent: <=50K
| occupation = Transport-moving
| | marital-status = Married-civ-spouse
| | | race = White: <=50K
| | | race = Asian-Pac-Islander: <=50K
| | marital-status = Divorced: <=50K
| | marital-status = Never-married: <=50K
| occupation = Protective-serv
| | race = White: <=50K
| | race = Amer-Indian-Eskimo: <=50K
| | race = Black: >50K
education = Prof-school
| workclass = Private: >50K
| workclass = Self-emp-not-inc: <=50K
| workclass = Self-emp-inc: >50K
| workclass = State-gov: >50K
education = Assoc-acdm
| occupation = Tech-support
| | relationship = Wife: <=50K
| | relationship = Husband: >50K
| occupation = Craft-repair: >50K
| occupation = Exec-managerial: <=50K
| occupation = Prof-specialty
| | marital-status = Married-civ-spouse: >50K
| | marital-status = Never-married: <=50K
| occupation = Machine-op-inspct: <=50K
| occupation = Adm-clerical
| | relationship = Wife: <=50K
| | relationship = Husband: >50K
| | relationship = Not-in-family: <=50K
| occupation = Farming-fishing: <=50K
| occupation = Transport-moving: <=50K
education = Assoc-voc
| occupation = Tech-support: <=50K
| occupation = Craft-repair: <=50K
| occupation = Other-service: <=50K
| occupation = Sales: >50K
| occupation = Exec-managerial
| | marital-status = Married-civ-spouse: >50K
| | marital-status = Divorced: <=50K
| occupation = Prof-specialty: <=50K
| occupation = Handlers-cleaners: <=50K
| occupation = Adm-clerical: <=50K
| occupation = Farming-fishing: <=50K
| occupation = Transport-moving: >50K
| occupation = Protective-serv: >50K
education = 9th: <=50K
education = 7th-8th: <=50K
education = 12th: <=50K
education = Masters
| relationship = Wife
| | occupation = Exec-managerial: <=50K
| | occupation = Prof-specialty: >50K
| relationship = Own-child: <=50K
| relationship = Husband: >50K
| relationship = Not-in-family
| | occupation = Sales: >50K
| | occupation = Exec-managerial: >50K
| | occupation = Prof-specialty
| | | workclass = Private: >50K
| | | workclass = State-gov: <=50K
| | occupation = Adm-clerical: <=50K
| relationship = Unmarried
| | workclass = Self-emp-inc: >50K
| | workclass = Local-gov: <=50K
education = 1st-4th: <=50K
education = 10th
| occupation = Craft-repair: <=50K
| occupation = Other-service: <=50K
| occupation = Prof-specialty: <=50K
| occupation = Handlers-cleaners: <=50K
| occupation = Transport-moving
| | marital-status = Married-civ-spouse: >50K
| | marital-status = Never-married: <=50K
education = Doctorate
| race = White: <=50K
| race = Asian-Pac-Islander: >50K
education = 5th-6th: <=50K
education = Preschool: <=50K
Time taken to build model: 0.03 seconds
=== Stratified cross-validation ===
=== Summary ===
Correctly Classified Instances 276 69 %
Incorrectly Classified Instances 64 16 %
Kappa statistic 0.4475
Mean absolute error 0.1934
Root mean squared error 0.4059
Relative absolute error 67.6175 %
Root relative squared error 108.2648 %
UnClassified Instances 60 15 %
Total Number of Instances 400
=== Detailed Accuracy By Class ===
TP Rate FP Rate Precision Recall F-Measure Class
0.592 0.13 0.545 0.592 0.568 >50K
0.87 0.408 0.89 0.87 0.88 <=50K
=== Confusion Matrix ===
a b <-- classified as
42 29 | a = >50K
35 234 | b = <=50K
Если удалить все строки со значением null (не встречающаяся комбинация параметров), то останется набор правил, по которым можно классифицировать объекты (в листинге 4 все строки со значением null удалены). Можно увидеть цепочки для которых в итоге будет >50K или <=50K, т.е. проходя по определенной цепочке устанавливаются условия, при которых экземпляр соответствует доходу >50K либо <=50K.
Анализ:
Если education = Bachelors и relationship = Wife и workclass = Local-gov и native-country = United-States то <=50K.
Если education = Bachelors и relationship = Wife и workclass = Local-gov и native-country = Germany то >50K.
Если education = Bachelors и relationship = Wife и workclass = Private или State-gov то >50K.
Если education = Bachelors и relationship = Own-child и workclass = Private и occupation = Prof-specialty то <=50K.
Если education = Bachelors и relationship = Own-child и workclass = Private и occupation = Exec-managerial то >50K.
Если education = Bachelors и relationship = Own-child и workclass = Self-emp-not-inc или State-gov то <=50K.
Если education = Bachelors и relationship = Own-child и workclass = Local-gov то >50K.
Если education = Bachelors и relationship = Husband и occupation = Sales и workclass = Self-emp-not-inc то <=50K.
Если education = Bachelors и relationship = Husband и occupation = Sales и workclass = Private или Self-emp-inc то >50K.
Если education = Bachelors и relationship = Husband и occupation = Prof-specialty и workclass = Private и native-country = Italy то <=50K.
Если education = Bachelors и relationship = Husband и occupation = Prof-specialty и workclass = Private и native-country = United-States то >50K.
Если education = Bachelors и relationship = Husband и occupation = Prof-specialty и workclass = Self-emp-not-inc или State-gov то <=50K.
Если education = Bachelors и relationship = Husband и occupation = Prof-specialty и workclass = Federal-gov то >50K.
Если education = Bachelors и relationship = Husband и occupation = Adm-clerical или Transport-moving то <=50K.
Если education = Bachelors и relationship = Husband и occupation = Craft-repair или Exec-managerial или Handlers-cleaners или Farming-fishing или Protective-serv то >50K.
Если education = Bachelors и relationship = Not-in-family и occupation = Tech-support или Craft-repair или Other-service или Exec-managerial или Prof-specialty или Machine-op-inspct или Transport-moving то <=50K.
Если education = Bachelors и relationship = Not-in-family и occupation = Sales или Adm-clerical то >50K.
Если education = Bachelors и relationship = Other-relative или Unmarried то <=50K.
Если education = Some-college и marital-status = Married-civ-spouse и occupation = Craft-repair и workclass = Private или Federal-gov то <=50K.
Если education = Some-college и marital-status = Married-civ-spouse и occupation = Exec-managerial и workclass = Private и race = White или Black то <=50K.
Если education = Some-college и marital-status = Married-civ-spouse и occupation = Exec-managerial и workclass = Self-emp-inc то <=50K.
Если education = Some-college и marital-status = Married-civ-spouse и occupation = Prof-specialty и race = Black и workclass = Private то <=50K.
Если education = Some-college и marital-status = Married-civ-spouse и occupation = Prof-specialty и race = White то <=50K.
Если education = Some-college и marital-status = Married-civ-spouse и occupation = Prof-specialty и race = Amer-Indian-Eskimo то >50K.
Если education = Some-college и marital-status = Married-civ-spouse и occupation = Transport-moving и workclass = Self-emp-inc то <=50K.
Если education = Some-college и marital-status = Married-civ-spouse и occupation = Transport-moving и workclass = Self-emp-not-inc то >50K.
Если education = Some-college и marital-status = Married-civ-spouse и occupation = Other-service или Handlers-cleaners или Adm-clerical или Farming-fishing или Protective-serv то <=50K.
Если education = Some-college и marital-status = Married-civ-spouse и occupation = Tech-support или Machine-op-inspct то >50K.
Если education = Some-college и marital-status = Divorced или Never-married или Separated или Widowed или Married-spouse-absent то <=50K.
Если education = HS-grad и occupation = Craft-repair и relationship = Husband и workclass = Private и race = White и native-country = United-States или Germany то <=50K.
Если education = HS-grad и occupation = Craft-repair и relationship = Husband и workclass = Private и race = Amer-Indian-Eskimo то <=50K.
Если education = HS-grad и occupation = Craft-repair и relationship = Husband и workclass = Self-emp-not-inc или Self-emp-inc или Local-gov то <=50K.
Если education = HS-grad и occupation = Craft-repair и relationship = Not-in-family и workclass = Private то <=50K.
Если education = HS-grad и occupation = Craft-repair и relationship = Not-in-family и workclass = Self-emp-not-inc то >50K.
Если education = HS-grad и occupation = Craft-repair и relationship = Own-child или Other-relative или Unmarried то <=50K.
Если education = HS-grad и occupation = Sales и marital-status = Married-civ-spouse и workclass = Private то <=50K.
Если education = HS-grad и occupation = Sales и marital-status = Married-civ-spouse и workclass = Self-emp-inc то >50K.
Если education = HS-grad и occupation = Sales и marital-status = Never-married или Separated или Widowed то <=50K.
Если education = HS-grad и occupation = Exec-managerial и relationship = Own-child или Husband или Not-in-family то <=50K.
Если education = HS-grad и occupation = Exec-managerial и relationship = Wife то >50K.
Если education = HS-grad и occupation = Prof-specialty и relationship = Own-child или Husband или Not-in-family или Other-relative то <=50K.
Если education = HS-grad и occupation = Prof-specialty и relationship = Wife то >50K.
Если education = HS-grad и occupation = Handlers-cleaners и workclass = Private то <=50K.
Если education = HS-grad и occupation = Handlers-cleaners и workclass = Self-emp-not-inc то >50K.
Если education = HS-grad и occupation = Farming-fishing и workclass = Self-emp-not-inc и marital-status = Married-spouse-absent то <=50K.
Если education = HS-grad и occupation = Farming-fishing и workclass = Self-emp-not-inc и marital-status = Married-civ-spouse то >50K.
Если education = HS-grad и occupation = Farming-fishing и workclass = Private то <=50K.
Если education = HS-grad и occupation = Transport-moving и marital-status = Married-civ-spouse и race = White или Asian-Pac-Islander то <=50K.
Если education = HS-grad и occupation = Transport-moving и marital-status = Divorced или Never-married то <=50K.
Если education = HS-grad и occupation = Protective-serv и race = White или Amer-Indian-Eskimo то <=50K.
Если education = HS-grad и occupation = Protective-serv и race = Black то >50K.
Если education = HS-grad и occupation = Tech-support или Other-service или Machine-op-inspct или Adm-clerical то <=50K.
Если education = Prof-school и workclass = Self-emp-not-inc то <=50K.
Если education = Prof-school и workclass = Private или Self-emp-inc или State-gov то >50K.
Если education = Assoc-acdm и occupation = Tech-support и relationship = Wife то <=50K.
Если education = Assoc-acdm и occupation = Tech-support и relationship = Husband то >50K.
Если education = Assoc-acdm и occupation = Prof-specialty и marital-status = Never-married то <=50K.
Если education = Assoc-acdm и occupation = Prof-specialty и marital-status = Married-civ-spouse то >50K.
Если education = Assoc-acdm и occupation = Adm-clerical и relationship = Wife или Not-in-family то <=50K.
Если education = Assoc-acdm и occupation = Adm-clerical и relationship = Husband то >50K.
Если education = Assoc-acdm и occupation = Exec-managerial или Machine-op-inspct или Farming-fishing или Transport-moving то <=50K.
Если education = Assoc-acdm и occupation = Craft-repair то >50K.
Если education = Assoc-voc и occupation = Exec-managerial и marital-status = Divorced то <=50K.
Если education = Assoc-voc и occupation = Exec-managerial и marital-status = Married-civ-spouse то >50K.
Если education = Assoc-voc и occupation = Tech-support или Craft-repair или Other-service или Prof-specialty или Handlers-cleaners или Adm-clerical или Farming-fishing то <=50K.
Если education = Assoc-voc и occupation = Sales или Transport-moving или Protective-serv то >50K.
Если education = Masters и relationship = Wife и occupation = Exec-managerial то <=50K.
Если education = Masters и relationship = Wife и occupation = Prof-specialty то >50K.
Если education = Masters и relationship = Not-in-family и occupation = Prof-specialty и workclass = State-gov то <=50K.
Если education = Masters и relationship = Not-in-family и occupation = Prof-specialty и workclass = Private то >50K.
Если education = Masters и relationship = Not-in-family и occupation = Adm-clerical то <=50K.
Если education = Masters и relationship = Not-in-family и occupation = Sales или Exec-managerial то >50K.
Если education = Masters и relationship = Unmarried и workclass = Local-gov то <=50K.
Если education = Masters и relationship = Unmarried и workclass = Self-emp-inc то >50K.
Если education = Masters и relationship = Own-child то <=50K.
Если education = Masters и relationship = Husband то >50K.
Если education = 10th и occupation = Transport-moving и marital-status = Never-married то <=50K.
Если education = 10th и occupation = Transport-moving и marital-status = Married-civ-spouse то >50K.
Если education = 10th и occupation = Craft-repair или Other-service или Prof-specialty или Handlers-cleaners то <=50K.
Если education = Doctorate и race = White то <=50K.
Если education = Doctorate и race = Asian-Pac-Islander то >50K.
Если education = 11th или 9th или 7th-8th или 12th или 1st-4th или 5th-6th или Preschool то <=50K.
Классификация методом J4.8 (модификация С4.5).
Этот алгоритм также применяется к исходным данным без их изменения. Результатом его работы является дерево решений, которое можно увидеть в виде дерева(рисунок 4), и текста (листинг 4).
Листинг 4
=== Classifier model (full training set) ===
J48 pruned tree
------------------
education-num <= 11
| capital-gain <= 4101: <=50K (281.0/24.0)
| capital-gain > 4101: >50K (10.0/3.0)
education-num > 11
| marital-status = Married-civ-spouse: >50K (66.0/19.0)
| marital-status = Divorced
| | education-num <= 13: <=50K (9.0)
| | education-num > 13: >50K (3.0)
| marital-status = Never-married
| | capital-gain <= 7688
| | | sex = Female: <=50K (13.0)
| | | sex = Male
| | | | hours-per-week <= 42: <=50K (10.0/1.0)
| | | | hours-per-week > 42: >50K (2.0)
| | capital-gain > 7688: >50K (3.0)
| marital-status = Separated: <=50K (1.0)
| marital-status = Widowed: <=50K (2.0)
| marital-status = Married-spouse-absent: >50K (0.0)
| marital-status = Married-AF-spouse: >50K (0.0)
Number of Leaves : 13
Size of the tree : 20
Time taken to build model: 0.09 seconds
=== Stratified cross-validation ===
=== Summary ===
Correctly Classified Instances 339 84.75 %
Incorrectly Classified Instances 61 15.25 %
Kappa statistic 0.5425
Mean absolute error 0.225
Root mean squared error 0.3542
Relative absolute error 65.9156 %
Root relative squared error 85.8436 %
Total Number of Instances 400
=== Detailed Accuracy By Class ===
TP Rate FP Rate Precision Recall F-Measure Class
0.621 0.089 0.659 0.621 0.639 >50K
0.911 0.379 0.896 0.911 0.903 <=50K
=== Confusion Matrix ===
a b <-- classified as
54 33 | a = >50K
28 285 | b = <=50K
 Рисунок 3
В этом случае дерево имеет лучший вид, чем при использовании метода ID3. Это вызвано 2 улучшениями алгоритма, по сравнению с ID3:
Возможность работать не только с категориальными атрибутами, но также с числовыми.
После построения дерева происходит усечение его ветвей. Если получившееся дерево слишком велико, выполняется либо группировка нескольких узлов в один лист, либо замещение узла дерева нижележащим поддеревом. Перед операцией над деревом вычисляется ошибка правила классификации, содержащегося в рассматриваемом узле. Если после замещения (или группировки) ошибка не возрастает (и не сильно увеличивается энтропия), значит замену можно произвести без ущерба для построенной модели.
При этом достигается более высокий процент правильности классификации (78% против 69.5% у ID3). По результатам вывода, а точнее дереву можно увидеть достаточно логичную классификацию объектов.
Анализ:
В нашем случае получается что дальнейшее ветвление дерева происходит при marital-status = Married-civ-spouse, marital-status = Divorced и marital-status = Widowed, для остальных имеем статистику верной/неверной классификации:
marital-status = Never-married: <=50K (114.0/3.0)
marital-status = Separated: <=50K (7.0)
marital-status = Married-spouse-absent: <=50K (4.0)
Значения (х.х/у.у) характеризуют соответственно верную/неверную классификацию примеров. Они стоят после определения принадлежности к классу <=50K или >50K.
Анализ:
Чаловек, удовлетворяющий критериям:
семейное положение = Married-civ-spouse
| прирост капитала <= 4386
| | количество образований <= 6: имеет годовой доход <=50K
| | количество образований > 6
| | | возраст <= 32: имеет годовой доход <=50K
| | | возраст > 32
| | | | рабочих часов в неделю <= 32: имеет годовой доход <=50K
| | | | рабочих часов в неделю > 32
| | | | | рабочий класс = Private
| | | | | | занятость = Tech-support, Sales, Exec-managerial, Adm-clerical, Self-emp-inc, Federal-gov, Local-gov, Prof-specialty: имеет годовой доход >50K
| | | | | | занятость = Other-service, Farming-fishing, Transport-moving, Self-emp-not-inc: имеет годовой доход <=50K
| | | | | | занятость = Machine-op-inspct
| | | | | | | прирост капитала <= 2580: имеет годовой доход <=50K
| | | | | | | прирост капитала > 2580: имеет годовой доход >50K
| прирост капитала > 4386: имеет годовой доход >50K
семейное положение = Divorced
| количество образований <= 12: имеет годовой доход <=50K
| количество образований > 12
| | занятость = Exec-managerial: имеет годовой доход >50K
| | занятость = Prof-specialty
| | | рабочих часов в неделю <= 47: имеет годовой доход <=50K
| | | рабочих часов в неделю > 47: имеет годовой доход >50K
семейное положение = Never-married, Separated, Married-spouse-absent: имеет годовой доход <=50K
семейное положение = Widowed: имеет годовой доход >50K
Классификация методом 1R (в системе Weka называется OneRule).
Метод классификации 1R – один из самых простых и понятных методов классификации. Применяется как к числовым данным, которые разбиваются на промежутки, так и к данным типа nominal.
Листинг 5
=== Classifier model (full training set) ===
capital-gain:
< 4243.5 -> <=50K
>= 4243.5 -> >50K
(309/400 instances correct)
Time taken to build model: 0.02 seconds
=== Stratified cross-validation ===
=== Summary ===
Correctly Classified Instances 301 75.25 %
Incorrectly Classified Instances 99 24.75 %
Kappa statistic 0.0983
Mean absolute error 0.2475
Root mean squared error 0.4975
Relative absolute error 64.6102 %
Root relative squared error 113.7664 %
Total Number of Instances 400
=== Detailed Accuracy By Class ===
TP Rate FP Rate Precision Recall F-Measure Class
0.087 0.017 0.643 0.087 0.154 >50K
0.983 0.913 0.756 0.983 0.855 <=50K
=== Confusion Matrix ===
a b <-- classified as
9 94 | a = >50K
5 292 | b = <=50K
Пример вывода алгоритма представлен в листинге 5.
Метод выбирает переменные принимающие наибольшее возможное количество значений, для таких переменных ошибка и будет наименьшей. Так, например, для переменной по которой у каждого ключа свое уникальное значение ошибка будет равно нулю, но для таких переменных правила бесполезны. В нашем случае такой переменной является Capital Gain. Соответственно после кросс-проверки точность результата также достаточно высока – 75,25 процент.
Анализ
Если прирост капитала < 4243.5 то годовой доход <=50K
Если прирост капитала >= 4243.5 то годовой доход >50K
Классификация методом SVM (в Weka называется SMO).
Для этого метода не требуется каких-либо преобразований исходной выборки.
Данный метод является алгоритмом классификации с использованием математических функций. Метод использует нелинейные математические функции. Номинальные данные преобразуются в числовые. Основная идея метода опорных векторов – перевод исходных векторов в пространство более высокой размерности и поиск максимальной разделяющей гиперплоскости в этом пространстве. Результат выполнения алгоритма представлен в листинге 6.
Листинг 6.
SMO
Classifier for classes: >50K, <=50K
BinarySMO
Machine linear: showing attribute weights, not support vectors.
-0.3401 * (normalized) age
+ 0.5086 * (normalized) workclass=Private
+ 0.6403 * (normalized) workclass=Self-emp-not-inc
+ -0.408 * (normalized) workclass=Self-emp-inc
+ -0.2808 * (normalized) workclass=Federal-gov
+ 0.0766 * (normalized) workclass=Local-gov
+ -0.5367 * (normalized) workclass=State-gov
+ -0.2626 * (normalized) fnlwgt
+ -0.6259 * (normalized) education=Bachelors
+ 0.1063 * (normalized) education=Some-college
+ 0.3107 * (normalized) education=11th
+ -0.0627 * (normalized) education=HS-grad
+ -0.0804 * (normalized) education=Prof-school
+ -0.6857 * (normalized) education=Assoc-acdm
+ 0.1835 * (normalized) education=Assoc-voc
+ 1.3199 * (normalized) education=9th
+ 0.0766 * (normalized) education=7th-8th
+ 0.0939 * (normalized) education=12th
+ -0.2795 * (normalized) education=Masters
+ 0 * (normalized) education=1st-4th
+ 0.6433 * (normalized) education=10th
+ -1 * (normalized) education=Doctorate
+ -1.4184 * (normalized) education-num
+ -1.0212 * (normalized) marital-status=Married-civ-spouse
+ -0.1019 * (normalized) marital-status=Divorced
+ 0.9589 * (normalized) marital-status=Never-married
+ 0.0211 * (normalized) marital-status=Separated
+ -0.638 * (normalized) marital-status=Widowed
+ 0.7812 * (normalized) marital-status=Married-spouse-absent
+ 0.4938 * (normalized) occupation=Tech-support
+ 0.302 * (normalized) occupation=Craft-repair
+ 0.3431 * (normalized) occupation=Other-service
+ -0.5918 * (normalized) occupation=Sales
+ -0.7765 * (normalized) occupation=Exec-managerial
+ -0.0066 * (normalized) occupation=Prof-specialty
+ 0 * (normalized) occupation=Handlers-cleaners
+ 0.3934 * (normalized) occupation=Machine-op-inspct
+ 0.108 * (normalized) occupation=Adm-clerical
+ -0.4923 * (normalized) occupation=Farming-fishing
+ 0.2268 * (normalized) occupation=Transport-moving
+ -0.6003 * (normalized) relationship=Wife
+ 0.4194 * (normalized) relationship=Own-child
+ 0.1739 * (normalized) relationship=Husband
+ 0.3034 * (normalized) relationship=Not-in-family
+ -0.5446 * (normalized) relationship=Other-relative
+ 0.2483 * (normalized) relationship=Unmarried
+ -0.5677 * (normalized) race=White
+ -0.0949 * (normalized) race=Asian-Pac-Islander
+ 0.3165 * (normalized) race=Amer-Indian-Eskimo
+ 0.346 * (normalized) race=Black
+ -0.1515 * (normalized) sex
+ -1.7759 * (normalized) capital-gain
+ -0.8419 * (normalized) capital-loss
+ -1.0458 * (normalized) hours-per-week
+ -0.4004 * (normalized) native-country=United-States
+ -0.5991 * (normalized) native-country=Canada
+ 0.7811 * (normalized) native-country=Germany
+ -0.1682 * (normalized) native-country=India
+ 0.4818 * (normalized) native-country=Japan
+ 0.8674 * (normalized) native-country=South
+ -0.2758 * (normalized) native-country=China
+ -0.5446 * (normalized) native-country=Philippines
+ 0.432 * (normalized) native-country=Italy
+ -1 * (normalized) native-country=Jamaica
+ 0.426 * (normalized) native-country=Mexico
+ 3.5477
Number of kernel evaluations: 41621 (91.842% cached)
Time taken to build model: 0.42 seconds
=== Stratified cross-validation ===
=== Summary ===
Correctly Classified Instances 330 82.5 %
Incorrectly Classified Instances 70 17.5 %
Kappa statistic 0.5274
Mean absolute error 0.175
Root mean squared error 0.4183
Relative absolute error 45.684 %
Root relative squared error 95.6633 %
Total Number of Instances 400
=== Detailed Accuracy By Class ===
TP Rate FP Rate Precision Recall F-Measure Class
0.612 0.101 0.677 0.612 0.643 >50K
0.899 0.388 0.87 0.899 0.884 <=50K
=== Confusion Matrix ===
a b <-- classified as
63 40 | a = >50K
30 267 | b = <=50K
На выводе алгоритма показываются веса для всех возможных атрибутов, при этом заметна задержка его вывода из-за проведения расчетов. Процент верной классификации оказывается достаточно высоким – 82,5%, а средняя ошибка классификатора наоборот, оказывается минимальной среди всех рассмотренных методов.
В итоге вывод данного алгоритма представлен в виде вектора n-мерного пространства. Цифры указанные в выводе – коэффициенты задающие плоскость, разделяющую исходные данные на классы.
|
