You are required to develop a Python program to analyze multilingual customer feedback stored in a CSV file. The program should utilize the pre-trained DistilBERT multilingual model to predict the sentiment of each feedback (e.g., Positive, Neutral, or Negative). Additionally, your program should summarize the sentiment distribution and compile the overall statistics into a JSON file.
Tasks
Complete the following tasks:
1. Load the Data:
- Load the customer feedback data from a CSV file located at
feedback.csv. - Ensure the data is read correctly into a pandas DataFrame. The CSV file will contain two columns:
id(Unique identifier) andfeedback(The customer feedback text).
2. Initialize Model and Tokenizer:
- Use the Hugging Face
transformerslibrary to load the pre-trained DistilBERT model and the corresponding tokenizer for multilingual text processing.
3. Predict Sentiment:
- Tokenize each feedback using the loaded tokenizer.
- Compute embeddings by passing the tokenized feedback through the model.
- Classify the sentiment as Positive, Neutral, or Negative using cosine similarity with predefined sentiment vectors:
- Positive sentiment vector:
[0.9, 0.8, 0.85] - Neutral sentiment vector:
[0.5, 0.5, 0.5] - Negative sentiment vector:
[0.1, 0.2, 0.15]
- Positive sentiment vector:
4. Compile Aggregate Statistics:
- Calculate the total number of feedbacks processed.
- Compute the percentage distribution of each sentiment category (Positive, Neutral, Negative).
- Identify the top 3 longest feedbacks by word count.
5. Save Results in JSON:
- Compile the results into a dictionary with keys:
total_feedbacks,sentiment_distribution,longest_feedbacks. - Save the results to a file named
sentiment_results.json