当前位置：首页 > news >正文

做网站优化的关键词怎么设置/2022黄页全国各行业

news 2025/8/16 21:45:47

做网站优化的关键词怎么设置,2022黄页全国各行业,网络工作室营业执照,wordpress 课程主题目录相关信息1 赛题2 数据分析2.1 加载数据2.2 查看缺失值2.3 标签分布2.4 文本长度2.5 标题摘要合并后字符长度3 总结相关信息【NLP】讯飞英文学术论文分类挑战赛Top10开源多方案–1 赛后总结与分析【NLP】讯飞英文学术论文分类挑战赛Top10开源多方案–2 数据分析【NLP】讯飞…

1 赛题

比赛官网

本次赛题希望参赛选手利用论文信息：论文id、标题、摘要，划分论文具体类别。
赛题样例（使用\t分隔）：
paperid：9821
title：Calculation of prompt diphoton production cross sections at Tevatron and LHC energies
abstract：A fully differential calculation in perturbative quantum chromodynamics is presented for the production of massive photon pairs at hadron colliders. All next-to-leading order perturbative contributions from quark-antiquark, gluon-(anti)quark, and gluon-gluon subprocesses are included, as well as all-orders resummation of initial-state gluon radiation valid at next-to-next-to-leading logarithmic accuracy.
categories：hep-ph
评估指标
本次竞赛的评价标准采用准确率指标，最高分为1。计算方法参考https://scikit-learn.org/stable/modules/generated/sklearn.metrics.accuracy_score.html，评估代码参考

from sklearn.metrics import accuracy_score
y_pred = [0, 2, 1, 3]
y_true = [0, 1, 2, 3]
accuracy_score(y_true, y_pred)

2 数据分析

github代码下载

2.1 加载数据

import re
import numpy as np
import pandas as pdtrain = pd.read_csv('./data/train.csv', sep='\t')
test = pd.read_csv('./data/test.csv', sep='\t')
sub = pd.read_csv('./data/sample_submit.csv')
# 查看前5行数据
train.head(5)

在这里插入图片描述
分别有四个属性，paperid、titile、abstract、categories

# 数据大小
train.shape,test.shape,sub.shape
#((50000, 4), (10000, 3), (10000, 2))

2.2 查看缺失值

# 查看训练集缺失值
train.isnull().sum()
# 查看测试集缺失值
test.isnull().sum()
# 查看训练集数据信息
train.info()
# 查看测试集数据信息
test.info()

paperid 0
title 0
abstract 0
categories 0
dtype: int64

paperid 0
title 0
abstract 0
dtype: int64

在这里插入图片描述

# 查看数据集中间有无空行
print("train null nums")
print(train.shape[0]-train.count())
print("test null nums")
print(test.shape[0]-test.count())

无缺失值，无空行，训练集有5W行，测试集有1W行

2.3 标签分布

train['categories'].value_counts()
# # 标签类别数 39
len(train['categories'].value_counts())# 绘制直方图
sns.countplot(train.categories)
plt.xlabel('categories count')

在这里插入图片描述
大类别只有CS一种，子类别有39类。数据分布不均衡

2.4 文本长度

# 训练集
titlelen1 = []
abstractlen1 = []for i in range(len(train)):titletext = train['title'][i]abstracttext = train['abstract'][i]titlelen1.append(len(tokenizer.tokenize(titletext)))abstractlen1.append(len(tokenizer.tokenize(abstracttext)))
train['titlelen'] = titlelen1
train['abstractlen'] = abstractlen1x1 = train['titlelen'].describe()
x2 = train['abstractlen'].describe()# 测试集
titlelen2 = []
abstractlen2 = []for i in range(len(test)):titletext = test['title'][i]abstracttext = test['abstract'][i]titlelen2.append(len(tokenizer.tokenize(titletext)))abstractlen2.append(len(tokenizer.tokenize(abstracttext)))
test['titlelen'] = titlelen2
test['abstractlen'] = abstractlen2x3 = test['titlelen'].describe()
x4 = test['abstractlen'].describe()

训练集

count 50000.00000
mean 10.60276
std 4.06394
min 1.00000
25% 8.00000
50% 10.00000
75% 13.00000
max 44.00000
Name: titlelen, dtype: float64

测试集

count 50000.00000
mean 186.88670
std 71.31268
min 5.00000
25% 137.00000
50% 182.00000
75% 231.00000
max 629.00000
Name: abstractlen, dtype: float64

训练集标题平均10个单词，摘要平均186个单词
测试集标题平均10个单词，测试集摘要平均186个单词

2.5 标题摘要合并后字符长度

train['text_len']=train['text'].map(len)
test['text_len']=test['text'].map(len)
train['text'].map(len).describe()test['text'].map(len).describe()
# 绘制句子长度分布图
train['text_len'].plot(kind='kde')
test['text_len'].plot(kind='kde')

count 50000.00000
mean 1131.28478
std 387.14365
min 69.00000
25% 860.00000
50% 1117.00000
75% 1393.00000
max 3713.00000
Name: text, dtype: float64

count 10000.000000
mean 1127.097700
std 388.662603
min 74.000000
25% 855.750000
50% 1111.000000
75% 1385.250000
max 3501.000000
Name: text, dtype: float64

在这里插入图片描述