Verify that for some eyes verifiy that psuedo comes to understand the next part of the dragon. I am creating several threads to implement some inproc functions as soon as possible. This idea is to lay the thread in the master loop, so the app will run threads simultaneously in a parallel /
code - write file names from each file in the name of each file, each file A queue for the name- spawn thread, where each thread waits for value / data from the queue - thread mercury function then handles actual processing based on the file included through the "execfile" function ... # System module queue importing queue thread importing thread import time # local module #import feedparser # set some global variable apes = quo () # will need more than # this matches the number of those files Packet #degroup to create the name of the file to join # urldir # Num_fetch_threads = 200 DF Threadpers (Q) # Different Component Line = q.get () College, Level, Packet = Disruption (Line) # College + "_" + level + "_p S. "Execfile (fname) q.task_done () #setup Mast Er Loop, while True time.sleep (2) #file # Get files from the setup thread filelist =" ls / urldir "if filelist foreach file_list in filelist: Worker = Thread (target = threadparas, args = (apu,) worker .start () # again, get # files from fear #setup queue file list = "ls / urldir" in file name foreach file__: Queue apule.poot (File_) file name # Wait for the queue now empty, it indicates that we processed on # of all downloads Ia is # about this part do not care # print "*** The main thread wait '# Apakway. Thoughts / comments / indicators are appreciated ...
thanks
If I consider this right: you are born many people threads to speed things up faster.
>
This works only when the main part of the work done in each thread is done without GIL, so if there is a lot of wait for data from network, disk or something like this, then it is a The Chchha might consider. If each task is using a lot of CPUs, then it will be very good like a core 1-CPU machine and you can sequence them too.
I should say that what I wrote is true for CPython, but it is not necessary for gastron / ironpathon. Also, I should add that if you have to use more CPU / core then Module which can help.
Comments
Post a Comment