注册
资讯
学习
基础知识 理论实践 学习教程
企业 招聘
下载
模型 数据集
AI工具箱
企业服务
EVA 智能HR 视频AI
ChatGPT 人工智能应用 人工智能未来 计算机视觉
热门新闻
机器人 康奈尔大学

康奈尔大学推出新的社交机器人，更柔软，更人性化
行业 人工智能

为什么数据污染对LLM来说是一个大问题
行业 人工智能

所以，能动手就别吵吵了
常用工具
ChatGPT

OpenAI旗下AI对话工具
Midjourney

AI图像和插画生成工具，测试测试测试测试测试测测试
Stable Diffusion

StabilityAI推出的文本到图像生成AI
GitHub Copilot

GitHub AI编程工具
文心一言

百度推出的基于文心大模型的AI对话互动工具
讯飞星火认知大模型

科大讯飞推出的类ChatGPT的讯飞星火认知大模型
心流

阿里旗下推出的AI搜索助手

数据集:

bigcode/github-commits-diff

数据集介绍 文件清单
英文

进行中

Name Megabytes % of total Samples % of total
total 3709175.78 100.0% 57700105 100.0%
json 583293.816 15.7257% 3495038 6.0572%
xml 279208.676 7.5275% 1923159 3.333%
text 270662.596 7.2971% 1389525 2.4082%
javascript 262824.844 7.0858% 5401937 9.3621%
objective-c++ 239009.3 6.4437% 32227 0.0559%
python 234311.564 6.3171% 6189601 10.7272%
c 200876.804 5.4157% 2779478 4.8171%
c++ 186585.256 5.0304% 2402294 4.1634%
markdown 171849.952 4.6331% 7645354 13.2502%
java 127103.448 3.4267% 3744377 6.4894%
html 105305.284 2.839% 2366841 4.102%
yaml 100466.64 2.7086% 2592787 4.4936%
go 86444.624 2.3306% 1183612 2.0513%
csv 82946.192 2.2362% 79268 0.1374%
php 74961.64 2.021% 2555419 4.4288%
jupyter-notebook 66854.08 1.8024% 94000 0.1629%
gettext-catalog 62296.88 1.6795% 168327 0.2917%
sql 56802.764 1.5314% 132772 0.2301%
unity3d-asset 39535.008 1.0659% 17867 0.031%
typescript 39254.804 1.0583% 572136 0.9916%
web-ontology-language 36435.464 0.9823% 7458 0.0129%
ruby 35830.74 0.966% 2928702 5.0757%
c# 33669.652 0.9077% 923157 1.5999%
nix 33547.92 0.9045% 221281 0.3835%
shell 25109.952 0.677% 1017977 1.7643%
perl 21148.928 0.5702% 374266 0.6486%
tex 17471.108 0.471% 89283 0.1547%
css 16306.632 0.4396% 548818 0.9512%
restructuredtext 15613.888 0.421% 494037 0.8562%
rust 15011.296 0.4047% 296214 0.5134%
groff 12020.188 0.3241% 32923 0.0571%
ini 8375.164 0.2258% 297100 0.5149%
scala 8325.96 0.2245% 316064 0.5478%
coffeescript 6795.14 0.1832% 292446 0.5068%
haskell 6306.12 0.17% 217325 0.3766%
swift 5902.716 0.1591% 319289 0.5534%
lua 5763.12 0.1554% 139091 0.2411%
svg 5645.44 0.1522% 27095 0.047%
gas 5585.384 0.1506% 15121 0.0262%
ocaml 5355.4 0.1444% 81360 0.141%
erlang 5043.32 0.136% 93685 0.1624%
makefile 4238.512 0.1143% 343379 0.5951%
asciidoc 4138.588 0.1116% 96671 0.1675%
emacs-lisp 3988.652 0.1075% 83228 0.1442%
scss 3944.936 0.1064% 288190 0.4995%
clojure 3523.408 0.095% 158674 0.275%
org 3126.22 0.0843% 30198 0.0523%
common-lisp 2954.904 0.0797% 74628 0.1293%
diff 2586.048 0.0697% 21021 0.0364%
groovy 2569.14 0.0693% 110057 0.1907%
html+erb 2450.676 0.0661% 225379 0.3906%
nesc 2439.564 0.0658% 473 0.0008%
dart 2395.796 0.0646% 56873 0.0986%
powershell 2289.276 0.0617% 55381 0.096%
f# 2289.236 0.0617% 66840 0.1158%
dm 2223.144 0.0599% 55584 0.0963%
kotlin 2219.248 0.0598% 124266 0.2154%
pascal 2194.676 0.0592% 42511 0.0737%
jsx 2124.744 0.0573% 139148 0.2412%
viml 1948.208 0.0525% 74062 0.1284%
actionscript 1844.148 0.0497% 28819 0.0499%
cython 1736.588 0.0468% 25927 0.0449%
turtle 1698.948 0.0458% 3882 0.0067%
less 1616.564 0.0436% 88634 0.1536%
mathematica 1475.044 0.0398% 925 0.0016%
xslt 1441.456 0.0389% 27956 0.0485%
scheme 1249.244 0.0337% 30546 0.0529%
perl6 1223.16 0.033% 12167 0.0211%
edn 1186.94 0.032% 2289 0.004%
fortran 1178.548 0.0318% 13463 0.0233%
java-server-pages 1173.072 0.0316% 53574 0.0928%
standard-ml 1133.476 0.0306% 20097 0.0348%
cmake 1132.068 0.0305% 58446 0.1013%
json5 1108.2 0.0299% 1827 0.0032%
vala 1104.512 0.0298% 14822 0.0257%
vue 1093.8 0.0295% 68967 0.1195%
freemarker 1032.332 0.0278% 36216 0.0628%
graphql 1004.844 0.0271% 2009 0.0035%
twig 958.96 0.0259% 39588 0.0686%
tcl 869.832 0.0235% 16407 0.0284%
pod 859.016 0.0232% 14922 0.0259%
dockerfile 849.728 0.0229% 259379 0.4495%
yacc 845.704 0.0228% 8230 0.0143%
postscript 800.728 0.0216% 903 0.0016%
racket 796.64 0.0215% 16615 0.0288%
eagle 785.684 0.0212% 2237 0.0039%
haxe 772.896 0.0208% 28447 0.0493%
julia 752.068 0.0203% 22695 0.0393%
handlebars 740.816 0.02% 49842 0.0864%
smarty 720.944 0.0194% 41065 0.0712%
visual-basic 681.516 0.0184% 10511 0.0182%
literate-haskell 673.74 0.0182% 10729 0.0186%
smalltalk 665.892 0.018% 11741 0.0203%
isabelle 655.82 0.0177% 8359 0.0145%
nimrod 652.86 0.0176% 12023 0.0208%
zig 621.384 0.0168% 4290 0.0074%
m4 603.584 0.0163% 12465 0.0216%
max 603.56 0.0163% 2259 0.0039%
elixir 558.116 0.015% 35473 0.0615%
mako 543.012 0.0146% 8943 0.0155%
arduino 534.176 0.0144% 32350 0.0561%
jade 531.4 0.0143% 46993 0.0814%
haml 502.012 0.0135% 74792 0.1296%
elm 481.968 0.013% 18542 0.0321%
purebasic 474.276 0.0128% 36 0.0001%
coldfusion 470.78 0.0127% 9263 0.0161%
lean 470.032 0.0127% 7507 0.013%
r 454.32 0.0122% 12858 0.0223%
cuda 437.668 0.0118% 11450 0.0198%
textile 425.116 0.0115% 18491 0.032%
robotframework 421.612 0.0114% 9211 0.016%
abap 409.62 0.011% 1955 0.0034%
rdoc 397.028 0.0107% 38760 0.0672%
llvm 382.2 0.0103% 10727 0.0186%
ada 380.7 0.0103% 13258 0.023%
batchfile 372.16 0.01% 43674 0.0757%
qml 361.452 0.0097% 19360 0.0336%
jasmin 359.82 0.0097% 4782 0.0083%
assembly 343.62 0.0093% 8126 0.0141%
g-code 334.964 0.009% 3690 0.0064%
cucumber 331.38 0.0089% 26677 0.0462%
html+php 323.348 0.0087% 18381 0.0319%
kicad 321.936 0.0087% 759 0.0013%
api-blueprint 317.852 0.0086% 4765 0.0083%
eiffel 311.48 0.0084% 373 0.0006%
toml 292.676 0.0079% 63517 0.1101%
modelica 284.616 0.0077% 2611 0.0045%
bitbake 277.576 0.0075% 43239 0.0749%
lex 275.96 0.0074% 705 0.0012%
stylus 273.056 0.0074% 21967 0.0381%
protocol-buffer 254.124 0.0069% 9202 0.0159%
unknown 252.228 0.0068% 30570 0.053%
nit 244.54 0.0066% 4951 0.0086%
factor 241.192 0.0065% 15378 0.0267%
xs 239.04 0.0064% 3215 0.0056%
sass 230.648 0.0062% 23144 0.0401%
parrot-internal-representation 230.196 0.0062% 6231 0.0108%
html+django 217.04 0.0059% 10535 0.0183%
mediawiki 214.324 0.0058% 10188 0.0177%
logos 212.296 0.0057% 1733 0.003%
genshi 209.3 0.0056% 956 0.0017%
coldfusion-cfc 208.164 0.0056% 4410 0.0076%
xtend 179.544 0.0048% 7775 0.0135%
sqf 168.656 0.0045% 7778 0.0135%
vhdl 155.948 0.0042% 2185 0.0038%
antlr 143.548 0.0039% 3651 0.0063%
systemverilog 140.192 0.0038% 3944 0.0068%
hcl 136.752 0.0037% 13379 0.0232%
asp 136.104 0.0037% 4286 0.0074%
nsis 129.124 0.0035% 4048 0.007%
inform-7 120.188 0.0032% 184 0.0003%
slim 119.036 0.0032% 18726 0.0325%
groovy-server-pages 117.368 0.0032% 6695 0.0116%
ceylon 116.144 0.0031% 7256 0.0126%
fish 111.28 0.003% 15351 0.0266%
processing 108.58 0.0029% 5912 0.0102%
component-pascal 105.5 0.0028% 43 0.0001%
lasso 104.168 0.0028% 67 0.0001%
glsl 99.488 0.0027% 9478 0.0164%
saltstack 98.196 0.0026% 12314 0.0213%
xbase 94.424 0.0025% 1670 0.0029%
autohotkey 94.22 0.0025% 1452 0.0025%
liquid 93.792 0.0025% 2651 0.0046%
purescript 92.412 0.0025% 5024 0.0087%
agda 92.06 0.0025% 4956 0.0086%
inno-setup 91.36 0.0025% 3014 0.0052%
oz 90.476 0.0024% 1551 0.0027%
chapel 89.62 0.0024% 26447 0.0458%
arc 87.212 0.0024% 758 0.0013%
opencl 86.432 0.0023% 2489 0.0043%
graphviz-dot 85.804 0.0023% 1525 0.0026%
pawn 85.424 0.0023% 580 0.001%
jsoniq 75.152 0.002% 1343 0.0023%
bluespec 72.38 0.002% 2500 0.0043%
smali 71.38 0.0019% 174 0.0003%
krl 69.868 0.0019% 1879 0.0033%
maple 68.284 0.0018% 1311 0.0023%
unrealscript 67.668 0.0018% 585 0.001%
ooc 63.188 0.0017% 3416 0.0059%
pure-data 62.624 0.0017% 603 0.001%
xquery 61.956 0.0017% 2237 0.0039%
digital-command-language 59.644 0.0016% 833 0.0014%
moonscript 59.208 0.0016% 1951 0.0034%
awk 57.176 0.0015% 2206 0.0038%
pike 52.872 0.0014% 1262 0.0022%
livescript 51.228 0.0014% 5194 0.009%
solidity 50.856 0.0014% 3689 0.0064%
monkey 48.256 0.0013% 1367 0.0024%
jsonld 48.012 0.0013% 462 0.0008%
zephir 42.684 0.0012% 1265 0.0022%
crystal 41.924 0.0011% 4217 0.0073%
rhtml 41.02 0.0011% 4551 0.0079%
stata 40.684 0.0011% 1344 0.0023%
idris 39.896 0.0011% 3025 0.0052%
raml 39.388 0.0011% 948 0.0016%
openscad 37.732 0.001% 2178 0.0038%
red 35.26 0.001% 1108 0.0019%
c2hs-haskell 34.472 0.0009% 1021 0.0018%
cycript 33.96 0.0009% 197 0.0003%
applescript 33.512 0.0009% 1304 0.0023%
mupad 32.488 0.0009% 178 0.0003%
literate-agda 31.384 0.0008% 567 0.001%
boo 31.172 0.0008% 26289 0.0456%
sourcepawn 29.528 0.0008% 717 0.0012%
qmake 29.508 0.0008% 3632 0.0063%
ragel-in-ruby-host 28.296 0.0008% 888 0.0015%
io 27.952 0.0008% 1247 0.0022%
desktop 27.648 0.0007% 5021 0.0087%
propeller-spin 26.772 0.0007% 625 0.0011%
thrift 26.748 0.0007% 1007 0.0017%
volt 25.052 0.0007% 1660 0.0029%
xproc 24.212 0.0007% 914 0.0016%
igor-pro 23.748 0.0006% 388 0.0007%
lolcode 23.74 0.0006% 24861 0.0431%
html+eex 21.412 0.0006% 2100 0.0036%
logtalk 20.428 0.0006% 1035 0.0018%
mirah 20.104 0.0005% 706 0.0012%
gnuplot 19.676 0.0005% 889 0.0015%
literate-coffeescript 19.016 0.0005% 1041 0.0018%
jflex 18.608 0.0005% 555 0.001%
emberscript 18.392 0.0005% 1024 0.0018%
cobol 17.0 0.0005% 24953 0.0432%
yang 16.94 0.0005% 597 0.001%
rebol 16.468 0.0004% 239 0.0004%
linker-script 16.084 0.0004% 1604 0.0028%
cartocss 15.916 0.0004% 555 0.001%
urweb 13.068 0.0004% 304 0.0005%
rmarkdown 13.032 0.0004% 750 0.0013%
darcs-patch 13.008 0.0004% 80 0.0001%
csound 12.852 0.0003% 229 0.0004%
squirrel 12.844 0.0003% 531 0.0009%
apl 12.56 0.0003% 586 0.001%
hlsl 12.168 0.0003% 1529 0.0026%
latte 11.888 0.0003% 1380 0.0024%
pony 11.836 0.0003% 624 0.0011%
ioke 10.86 0.0003% 373 0.0006%
hy 10.512 0.0003% 879 0.0015%
uno 10.356 0.0003% 628 0.0011%
pan 10.336 0.0003% 637 0.0011%
xojo 10.308 0.0003% 642 0.0011%
papyrus 10.256 0.0003% 130 0.0002%
stan 10.252 0.0003% 540 0.0009%
slash 9.904 0.0003% 640 0.0011%
supercollider 9.796 0.0003% 318 0.0006%
vcl 9.456 0.0003% 747 0.0013%
smt 9.032 0.0002% 117 0.0002%
glyph 8.948 0.0002% 7 0.0%
wisp 8.736 0.0002% 262 0.0005%
renpy 8.3 0.0002% 421 0.0007%
clips 7.728 0.0002% 450 0.0008%
dns-zone 7.56 0.0002% 54 0.0001%
sas 7.536 0.0002% 269 0.0005%
rouge 7.196 0.0002% 396 0.0007%
ec 7.032 0.0002% 94 0.0002%
dylan 6.82 0.0002% 280 0.0005%
tcsh 6.524 0.0002% 748 0.0013%
aspectj 6.332 0.0002% 451 0.0008%
netlogo 6.304 0.0002% 140 0.0002%
gap 6.096 0.0002% 46 0.0001%
fancy 5.952 0.0002% 675 0.0012%
coq 5.744 0.0002% 80 0.0001%
click 5.74 0.0002% 9 0.0%
capn-proto 5.644 0.0002% 330 0.0006%
flux 5.572 0.0002% 47 0.0001%
forth 5.512 0.0001% 265 0.0005%
ats 5.424 0.0001% 383 0.0007%
netlinx 5.172 0.0001% 144 0.0002%
clean 5.068 0.0001% 171 0.0003%
parrot-assembly 4.664 0.0001% 227 0.0004%
alloy 4.644 0.0001% 203 0.0004%
lfe 4.576 0.0001% 287 0.0005%
gdscript 4.488 0.0001% 460 0.0008%
augeas 4.444 0.0001% 395 0.0007%
sparql 4.404 0.0001% 1036 0.0018%
lilypond 4.308 0.0001% 265 0.0005%
scilab 4.088 0.0001% 375 0.0006%
autoit 4.06 0.0001% 279 0.0005%
myghty 3.864 0.0001% 105 0.0002%
blitzmax 3.74 0.0001% 220 0.0004%
creole 3.416 0.0001% 337 0.0006%
harbour 3.336 0.0001% 107 0.0002%
piglatin 3.168 0.0001% 513 0.0009%
opa 3.164 0.0001% 211 0.0004%
sage 3.032 0.0001% 414 0.0007%
ston 2.848 0.0001% 414 0.0007%
maxscript 2.8 0.0001% 47 0.0001%
lsl 2.68 0.0001% 74 0.0001%
gentoo-ebuild 2.576 0.0001% 601 0.001%
nu 2.38 0.0001% 170 0.0003%
bro 2.34 0.0001% 333 0.0006%
xc 2.02 0.0001% 88 0.0002%
j 1.808 0.0% 142 0.0002%
metal 1.724 0.0% 151 0.0003%
module-management-system 1.544 0.0% 91 0.0002%
webidl 1.508 0.0% 96 0.0002%
tea 1.468 0.0% 29 0.0001%
redcode 1.272 0.0% 149 0.0003%
shen 1.2 0.0% 71 0.0001%
pov-ray-sdl 1.136 0.0% 104 0.0002%
x10 1.008 0.0% 33 0.0001%
brainfuck 0.964 0.0% 167 0.0003%
ninja 0.952 0.0% 187 0.0003%
golo 0.896 0.0% 115 0.0002%
webassembly 0.86 0.0% 83 0.0001%
self 0.824 0.0% 15 0.0%
labview 0.808 0.0% 61 0.0001%
octave 0.804 0.0% 12 0.0%
pogoscript 0.804 0.0% 74 0.0001%
d 0.796 0.0% 20 0.0%
http 0.736 0.0% 140 0.0002%
ecl 0.664 0.0% 48 0.0001%
chuck 0.584 0.0% 99 0.0002%
gosu 0.524 0.0% 60 0.0001%
parrot 0.52 0.0% 17 0.0%
opal 0.472 0.0% 69 0.0001%
objective-j 0.456 0.0% 37 0.0001%
kit 0.412 0.0% 48 0.0001%
gams 0.376 0.0% 18 0.0%
prolog 0.276 0.0% 35 0.0001%
clarion 0.268 0.0% 13 0.0%
mask 0.252 0.0% 37 0.0001%
brightscript 0.244 0.0% 28 0.0%
scaml 0.184 0.0% 31 0.0001%
matlab 0.164 0.0% 29 0.0001%
idl 0.148 0.0% 1 0.0%
ags-script 0.124 0.0% 31 0.0001%
lookml 0.12 0.0% 10 0.0%
apacheconf 0.108 0.0% 59 0.0001%
oxygene 0.104 0.0% 9 0.0%
txl 0.096 0.0% 3 0.0%
grammatical-framework 0.088 0.0% 39 0.0001%
renderscript 0.064 0.0% 54 0.0001%
mtml 0.052 0.0% 13 0.0%
unified-parallel-c 0.052 0.0% 6 0.0%
dogescript 0.04 0.0% 10 0.0%
gentoo-eclass 0.04 0.0% 6 0.0%
zimpl 0.04 0.0% 7 0.0%
irc-log 0.036 0.0% 9 0.0%
fantom 0.028 0.0% 11 0.0%
numpy 0.028 0.0% 1 0.0%
cirru 0.024 0.0% 4 0.0%
xpages 0.024 0.0% 7 0.0%
nginx 0.02 0.0% 6 0.0%
objdump 0.02 0.0% 1 0.0%
python-traceback 0.02 0.0% 10 0.0%
realbasic 0.012 0.0% 1 0.0%
befunge 0.008 0.0% 2 0.0%
bison 0.008 0.0% 1 0.0%
m 0.008 0.0% 1 0.0%
omgrofl 0.008 0.0% 1 0.0%

作者:

bigcode

数据集大小:

68.98 GB

相关推荐