問題描述

我正在嘗試從

我對你的代碼添加了一些調(diào)整并運行測試如下:

• 代碼塊:

  從 selenium 導(dǎo)入 webdriver從 selenium.webdriver.common.by 導(dǎo)入從 selenium.webdriver.support.ui 導(dǎo)入 WebDriverWait從 selenium.webdriver.support 導(dǎo)入 expected_conditions 作為 EC選項 = webdriver.ChromeOptions()options.add_argument("開始最大化")options.add_experimental_option("excludeSwitches", ["enable-automation"])options.add_experimental_option('useAutomationExtension', False)driver = webdriver.Chrome(options=options, executable_path=r'C:UtilityBrowserDriverschromedriver.exe')driver.get('https://www1.nseindia.com/products/content/equities/equities/eq_monthly_statistics.htm')Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR,"select#h_filetype")))).select_by_visible_text("資本市場")Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR,"select#yearField")))).select_by_visible_text("2019-2020")WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.getdata-button#get[type='image'][src^='/common/images/btn-get-data.gif']))).點??擊()

觀察

與您的觀察類似，我遇到了同樣的障礙，但沒有結(jié)果如下:

深入研究

似乎 Get Data 元素上的 click() 文本確實發(fā)生了.但是在檢查 webpage 的 驅(qū)動 WebDriver 實例，響應(yīng)被阻塞.

參考文獻(xiàn)

一些文檔:

• 機(jī)器人管理器
• 機(jī)器人經(jīng)理:基礎(chǔ)

tl;博士

幾個相關(guān)的討論:

• Selenium webdriver:修改導(dǎo)航器.webdriver 標(biāo)志以防止硒檢測
• 無法使用 Selenium 自動化 Chase網(wǎng)站登錄

I am trying to scrape data from here.

By clicking on the capital market and 2019-20 year. I want to click on Get data.

I have used following code:

driver = webdriver.Chrome(executable_path=chrome_path,options=chrome_options)

driver.get( nse_cash_keystats_page )


 driver.find_element_by_xpath( "http://select[@id='h_filetype']/option[text()='Capital Market ']" ).click()

driver.find_element_by_xpath( "http://select[@id='yearField']/option[text()='2019-2020']" ).click()

     downloadButton=WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.XPATH,'//input[@type="image"][@src="/common/images/btn-get-data.gif"]')))

driver.execute_script("arguments[0].click();", downloadButton)

By using the above code, I am able to click on Get DATA. But it is not showing output.

Please help me.Thanks in advance.

I took your code added a few tweaks and ran the test as follows:

• Code Block:

  from selenium import webdriver
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.support import expected_conditions as EC
  
  options = webdriver.ChromeOptions() 
  options.add_argument("start-maximized")
  options.add_experimental_option("excludeSwitches", ["enable-automation"])
  options.add_experimental_option('useAutomationExtension', False)
  driver = webdriver.Chrome(options=options, executable_path=r'C:UtilityBrowserDriverschromedriver.exe')
  driver.get('https://www1.nseindia.com/products/content/equities/equities/eq_monthly_statistics.htm')
  Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR,"select#h_filetype")))).select_by_visible_text("Capital Market ")
  Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR,"select#yearField")))).select_by_visible_text("2019-2020")
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.getdata-button#get[type='image'][src^='/common/images/btn-get-data.gif']"))).click()
  

Observation

Similar to your observation, I have hit the same roadblock with no results as follows:

Deep Dive

It seems the click() on the element with text as Get Data does happens. But while inspecting the DOM Tree of the webpage you will find that some of the <script> tag refers to JavaScripts having keyword akam. As an example:

• <script type="text/javascript" src="https://www1.nseindia.com/akam/11/52349752" defer=""></script>
• <noscript><img src="https://www1.nseindia.com/akam/11/pixel_52349752?a=dD01ZDZiMTA5OGQ0MDljYTYxN2RjMjc3MzBlN2YwMDQ0NjlkZDNiNTMzJmpzPW9mZg==" style="visibility: hidden; position: absolute; left: -999px; top: -999px;" /></noscript>

Which is a clear indication that the website is protected by Bot Manager an advanced bot detection service provided by Akamai and the response gets blocked.

Bot Manager

As per the article Bot Manager - Foundations:

Conclusion

So it can be concluded that the request for the data is detected as being performed by Selenium driven WebDriver instance and the response is blocked.

References

A couple of documentations:

• Bot Manager
• Bot Manager : Foundations

tl; dr

A couple of relevant discussions:

• Selenium webdriver: Modifying navigator.webdriver flag to prevent selenium detection
• Unable to use Selenium to automate Chase site login

這篇關(guān)于在 nseindia.com 上單擊獲取數(shù)據(jù)按鈕以獲取每月結(jié)算統(tǒng)計信息不會使用 Selenium 和 Python 獲取結(jié)果的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網(wǎng)！