For example, if you’re classifying types of cancer in the general population, many cancers are quite rare. K-NN (and Naive Bayes) outperform decision trees when it comes to rare occurrences. On the other hand, Naive Bayes does require training.ĥ. K-NN doesn’t require any training-you just load the dataset and off it runs. In comparison, K-NN doesn’t require that you know anything about the underlying probability distributions.Ĥ. Therefore, unless you know the probabilities and pdfs, use of the ideal Bayes is unrealistic. The algorithm compares all other classifiers against this ideal. ![]() Naive Bayes requires that you known the underlying probability distributions for categories. Naive Bayes will only work if the decision boundary is linear, elliptic, or parabolic. Naive Bayes can suffer from the zero probability problem when a particular attribute’s conditional probability equals zero, Naive Bayes will completely fail to produce a valid prediction. This could be fixed using a Laplacian estimator, but K-NN could end up being the easier choice.Ģ. If having conditional independence will highly negative affect classification, you’ll want to choose K-NN over Naive Bayes. For tasks like robotics and computer vision, Bayes outperforms decision trees. This method is not affected by the curse of dimensionality and l arge feature sets, while K-NN has problems with both.ĥ. In comparison, K-NN only has one option for tuning: the “ k”, or number of neighbors.Ĥ. A hyperparameter is a prior parameter that are tuned on the training set to optimize it. Naive Bayes offers you two hyperparameters to tune for smoothing: alpha and beta. Don’t discount K-NN when it comes to accuracy though as the value of k in K-NN increases, the error rate decreases until it reaches that of the ideal Bayes (for k→∞).ģ. In general, Naive Bayes is highly accurate when applied to big data. In comparison, k-nn is usually slower for large amounts of data, because of the calculations required for each new step in the process. If speed is important, choose Naive Bayes over K-NN.Ģ. Naive Bayes is a linear classifier while K-NN is not It tends to be faster when applied to big data. The math is complex, but the result is a process that’s highly accurate and fast-especially when you’re dealing with Big Data. Finally, you’ll want to dig into Naive Bayes. K-NN comes in a close second Although the math behind it is a little daunting, you can still create a visual of the nearest neighbor process to understand the process. It will give you a clear visual, and it’s ideal to get a grasp on what classification is actually doing. If you’re new to classification, a decision tree is probably your best starting point. Decision trees are easy to use for small amounts of classes. If you’re trying to decide between the three, your best option is to take all three for a test drive on your data, and see which produces the best results. Naive Bayes and K-NN, are both examples of supervised learning (where the data comes already labeled). ![]() That said, three popular classification methods- Decision Trees, k-NN & Naive Bayes-can be tweaked for practically every situation. In general, there isn’t a single “best” option for every situation. A myriad of options exist for classification.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |