네이버 블로그 크롤링

library(stringr)


getwd()


keyword<-"사당 맛집" ## keyword설정 
keyword_list<-c("사당 맛집","이태원 맛집","인프런")
getwd()



base_folder<-"D://인프런//crawling"

i<-1
for(i in 1:length(keyword_list)){

keyword<-keyword_list[i]
keyword<-iconv(keyword,from = "CP949",to="UTF-8")

if(is.na(keyword)){
  keyword<-keyword_list[i]
  keyword<-iconv(keyword,from = "CP949",to="UTF-8")
}


keyword2<-URLencode(keyword)

blog_num<-20 # 수집할 블로그의 갯수 
page_num<-round(blog_num/10)
final_id<-NULL
final_log<-NULL

for(k in 1:page_num){
page_index<-(k-1)*10 +1
url<-paste0("https://search.naver.com/search.naver?date_from=&date_option=0&date_to=&dup_remove=1&nso=&post_blogurl=&post_blogurl_without=&query=",keyword2,"&sm=tab_pge&srchby=all&st=sim&where=post&start=",page_index)

b<-readLines(url,encoding = "UTF-8")
b2<-b[str_detect(b,"tab_depth|value=\"tab_opt\">")]
b3<-str_split(b2,"sh_blog_title _sp_each_url _sp_each_title")
b4<-sapply(str_split(b3[[1]],"target=_blank"),function(x){x[1]})
b5<-str_sub(b4[2:length(b4)],9,end=-3)

naver_id<-str_sub(str_extract(b5,("(?<=com/).*(?=Redirect)")),end=-2)
naver_id<-naver_id[!is.na(naver_id)]
log_no<-sapply(str_split(b5,"logNo="),function(x){x[2]})
log_no<-log_no[!is.na(log_no)]
final_id<-c(final_id,naver_id)
final_log<-c(final_log,log_no)
cat("\n",k)
}



blog_url<-paste0("https://blog.naver.com/PostView.nhn?blogId=",final_id,"&logNo=",final_log,"&redirect=Dlog&widgetTypeCall=true&directAccess=false")


final_con<-c()
for(j in 1:length(blog_url)){
b<-readLines(blog_url[j],encoding="UTF-8")

b2<-b[which(str_detect(b,"

")):which(str_detect(b,"post_footer_contents"))]
head(b2)

b3<-paste(b2,collapse = "")
con<-gsub("  ","",str_trim(gsub("<.*?>","",b3)))
con<-gsub("\t","",con)
final_con[j]<-con

cat("\n",j)
}

final_con<-gsub("・","",final_con)

dir.create(paste0(base_folder,"//",keyword))
setwd(paste0(base_folder,"//",keyword))



write.csv(final_con,paste0(keyword,"blog_data.csv"),row.names=F)
cat("\n",keyword,"수집 완료")

}

인프런 강의 할인 쿠폰 코드입니다.

R로 하는 웹 크롤링 - 입문편(할인쿠폰코드) 236-69fc51d93a53 
https://www.inflearn.com/course/R-crawling# 

R로 하는 웹 크롤링 - 실전편 (할인쿠폰코드) 237-55c672aeb038 
https://www.inflearn.com/course/R-crawling-2# 

R로 하는 텍스트마이닝 (Top keyword부터 감성분석까지) (할인쿠폰코드) 238-c86096730061 
https://www.inflearn.com/course/R-text-mining#

 

R로 무작정 해보는 data science (할인쿠폰코드) 310-c6164f3d9be9 
https://www.inflearn.com/course/R-%EB%8D%B0%EC%9D%B4%ED%84%B0%EC%82%AC%EC%9D%B4%EC%96%B8%EC%8A%A4-%EB%9D%BC%EC%9D%B4%EB%B8%8C%EC%BD%94%EB%94%A9#

R로 무작정 해보는 data science - 인프런