Update README.md
Browse files
README.md
CHANGED
@@ -1,27 +1,62 @@
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
---
|
4 |
-
Bylastic: A Log Classifier Compatible with Elastic
|
5 |
-
Introduction
|
6 |
-
Bylastic is an advanced AI-based log classification model specifically designed to categorize records into three levels: ERROR, WARNING, and INFO. This model is fully optimized for integration with Elastic, offering an efficient and accurate solution for log management and analysis.
|
7 |
-
|
8 |
-
Key Features
|
9 |
-
Accurate Classification: Bylastic classifies logs into three critical categories: ERROR, WARNING, and INFO, helping to quickly identify problems, warnings, and general system information.
|
10 |
-
Full Elastic Compatibility: Designed to seamlessly integrate with Elastic, Bylastic facilitates data ingestion and analysis within the Elastic ecosystem.
|
11 |
-
High Performance: Optimized to process large volumes of logs, ensuring fast and efficient performance even in high-demand environments.
|
12 |
-
Easy Integration: Bylastic can be easily integrated into your existing log processing pipelines, reducing implementation time and associated costs.
|
13 |
-
How It Works
|
14 |
-
Bylastic utilizes advanced natural language processing (NLP) techniques, a branch of artificial intelligence (AI), to analyze and categorize logs. The model has been trained with a diverse set of log data, ensuring high accuracy in classification.
|
15 |
|
16 |
-
|
17 |
-
ERROR: Logs indicating critical failures or serious problems that require immediate attention.
|
18 |
-
WARNING: Logs indicating potential issues that could become errors if not properly managed.
|
19 |
-
INFO: Informational logs that provide details about the normal functioning of the system.
|
20 |
|
|
|
21 |
|
22 |
-
|
23 |
|
|
|
|
|
|
|
|
|
24 |
|
|
|
|
|
|
|
|
|
|
|
25 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/65bc645cb7db0ab095f10320/gpCmIOXPScF4R2LgMre0z.png)
|
26 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
27 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/65bc645cb7db0ab095f10320/tTaQ_H84CqPR0b3Sl0Thv.png)
|
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
---
|
4 |
+
# Bylastic: A Log Classifier Compatible with Elastic
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
5 |
|
6 |
+
## Introduction
|
|
|
|
|
|
|
7 |
|
8 |
+
**Bylastic** is an advanced AI-based log classification model specifically designed to categorize records into three levels: **ERROR**, **WARNING**, and **INFO**. Created by **Byviz**, this model is fully optimized for integration with Elastic, offering an efficient and accurate solution for log management and analysis.
|
9 |
|
10 |
+
## Key Features
|
11 |
|
12 |
+
- **Accurate Classification**: Bylastic classifies logs into three critical categories: **ERROR**, **WARNING**, and **INFO**, helping to quickly identify problems, warnings, and general system information.
|
13 |
+
- **Full Elastic Compatibility**: Designed to seamlessly integrate with Elastic, Bylastic facilitates data ingestion and analysis within the Elastic ecosystem.
|
14 |
+
- **High Performance**: Optimized to process large volumes of logs, ensuring fast and efficient performance even in high-demand environments.
|
15 |
+
- **Easy Integration**: Bylastic can be easily integrated into your existing log processing pipelines, reducing implementation time and associated costs.
|
16 |
|
17 |
+
### Precision Bylastic vs Bert
|
18 |
+
Precision in categorizing logs with example data, Bert cannot identify the categories
|
19 |
+
![image/png](https://cdn-uploads.huggingface.co/production/uploads/65bc645cb7db0ab095f10320/hpwBT4dKWt3rIKAjn4UM4.png)
|
20 |
+
|
21 |
+
## Requeriments Bylastic vs Bert
|
22 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/65bc645cb7db0ab095f10320/gpCmIOXPScF4R2LgMre0z.png)
|
23 |
|
24 |
+
|
25 |
+
## How It Works
|
26 |
+
|
27 |
+
Bylastic utilizes advanced natural language processing (NLP) techniques, a branch of artificial intelligence (AI), to analyze and categorize logs. The model has been trained with a diverse set of log data, ensuring high accuracy in classification.
|
28 |
+
|
29 |
+
### Log Categories
|
30 |
+
|
31 |
+
- **ERROR**: Logs indicating critical failures or serious problems that require immediate attention.
|
32 |
+
- **WARNING**: Logs indicating potential issues that could become errors if not properly managed.
|
33 |
+
- **INFO**: Informational logs that provide details about the normal functioning of the system.
|
34 |
+
|
35 |
+
## Integration with Elastic
|
36 |
+
|
37 |
+
Integrating Bylastic with Elastic is straightforward and direct. Here is a quick guide to integrate the model into your Elastic environment:
|
38 |
+
|
39 |
+
1. **Installation**: Download Bylastic from Hugging Face.
|
40 |
+
2. **Upload the Model**: Use `eland` to upload the model to your Elastic cluster.
|
41 |
+
3. **Create an inference pipeline in Elastic**
|
42 |
+
|
43 |
+
## Benefits
|
44 |
+
|
45 |
+
- **Improved Log Management**: Facilitates the identification and resolution of issues by automatically classifying logs.
|
46 |
+
- **Time Savings**: Reduces the time required to manually review and categorize logs.
|
47 |
+
- **Higher Accuracy**: Minimizes human errors in log classification.
|
48 |
+
- **Easy Integration**: Seamlessly integrates with Elastic, leveraging Elastic's advanced search and analysis capabilities.
|
49 |
+
|
50 |
+
|
51 |
+
# Load model directly
|
52 |
+
|
53 |
+
```python
|
54 |
+
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
55 |
+
|
56 |
+
tokenizer = AutoTokenizer.from_pretrained("byviz/bylastic_classification_logs")
|
57 |
+
model = AutoModelForSequenceClassification.from_pretrained("byviz/bylastic_classification_logs")
|
58 |
+
```
|
59 |
+
|
60 |
+
|
61 |
+
## Test with Elastic
|
62 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/65bc645cb7db0ab095f10320/tTaQ_H84CqPR0b3Sl0Thv.png)
|