How to Sow[] until I've Reap[]'d enough?Elements in `Reap` and `Sow`Reap, Sow with Parallelize: bad performance, why?Writing Faster Mathematica Code - Sow and Reap?Timing functions with Sow / Reap and AbsoluteTimingPoor performance from Manipulate and Sow-ReapBetter definitions of Reap and SowUnderstanding Sow and Reap documentationHow to use Reap and Sow instead of Append toReap and sow for BreadthFirstSearchI need an alternative to AppendTo using Reap and Sow
What DC should I use for someone trying to survive indefinitely solely with an alchemy jug as their only source of food and water? (survival campaign)
What do you call someone whose unmarried partner has died?
At what point in time would humans notice a 21st century satellite observing them?
Source of an alternate universe puzzle game with a cymbal-sporting singing monkey toy
Time machine in Interstellar movie
How to understand Generative Adversarial Networks Discriminative distribution?
On a naked chicken (no coating,batter) is there any benefit of double frying?
Is there any point in adding more than 6 months' runway in savings instead of investing everything after that?
How do you write an attribution if there's ambiguity over which version of Creative Commons applies?
My boss wants me to recreate everything I have done for my previous employer
How to use FDE without needing to share the encryption password
Can a 'Second Referendum' on Brexit contain three options?
Was there a clearly identifiable "first computer" to use or demonstrate the use of virtual memory?
Is this a pure function?
Passport expiration requirement for Jordan Visa
Why did the Bohr Model Successfully calculate some of the energy levels in hydrogen?
Why was the DC-9-80 so successful despite being obsolete almost from birth?
How to manage publications on local computer
Grep over multiple files redirecting to a different filename each time
How to talk about things I'm ashamed of, without showing this shame through non-verbal-communication?
Iodine tablet correct use and efficiency
How do I resolve science-based problems in my worldbuilding?
Why rounding odd font sizes to even?
Speaking German abroad and feeling condescended to when people speak English back to me
How to Sow[] until I've Reap[]'d enough?
Elements in `Reap` and `Sow`Reap, Sow with Parallelize: bad performance, why?Writing Faster Mathematica Code - Sow and Reap?Timing functions with Sow / Reap and AbsoluteTimingPoor performance from Manipulate and Sow-ReapBetter definitions of Reap and SowUnderstanding Sow and Reap documentationHow to use Reap and Sow instead of Append toReap and sow for BreadthFirstSearchI need an alternative to AppendTo using Reap and Sow
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty
margin-bottom:0;
.everyonelovesstackoverflowposition:absolute;height:1px;width:1px;opacity:0;top:0;left:0;pointer-events:none;
$begingroup$
I have a process that returns an unpredictable number of data points, and I'd like to run it repeatedly until I have a certain number of points.
My actual code is too complicated to use an illustration, so I wrote this toy example. fakeData[] will return 1-21 data points, and I want to run it until I have at least 100. But this code doesn't work because you can't take the so-far Length[] of a list that you're still building.
fakeData[n_] := RandomReal[1, 1 + RandomInteger[n]];
big = Reap[
While[Length[big] < 100, (* this doesn't work*)
Sow[fakeData[20]]]][[2, 1]]
I could just allocate 'big' as a Table with length 100 and copy each new small list into it, but then I'd have to discard some perfectly good data points I laboriously calculated, which is distasteful. Is there a better way?
sow-reap
asked Jul 23 at 18:01
Jerry GuernJerry Guern
2,11410 silver badges37 bronze badges
$endgroup$
add a comment
|
$begingroup$
I have a process that returns an unpredictable number of data points, and I'd like to run it repeatedly until I have a certain number of points.
My actual code is too complicated to use an illustration, so I wrote this toy example. fakeData[] will return 1-21 data points, and I want to run it until I have at least 100. But this code doesn't work because you can't take the so-far Length[] of a list that you're still building.
fakeData[n_] := RandomReal[1, 1 + RandomInteger[n]];
big = Reap[
While[Length[big] < 100, (* this doesn't work*)
Sow[fakeData[20]]]][[2, 1]]
I could just allocate 'big' as a Table with length 100 and copy each new small list into it, but then I'd have to discard some perfectly good data points I laboriously calculated, which is distasteful. Is there a better way?
sow-reap
asked Jul 23 at 18:01
Jerry GuernJerry Guern
2,11410 silver badges37 bronze badges
$endgroup$
$begingroup$
WillNestWhile[(Join[#, fakeData[20]]) &, , Length[#] < 100 &]work for you?
$endgroup$
– Shadowray
Jul 24 at 20:19
$begingroup$
@Shadowray That will work, but I've been told that using Join[] repeatedly is very, very slow because of all the recopying necessary every time you add a chunk.
$endgroup$
– Jerry Guern
Jul 25 at 0:05
add a comment
|
$begingroup$
I have a process that returns an unpredictable number of data points, and I'd like to run it repeatedly until I have a certain number of points.
My actual code is too complicated to use an illustration, so I wrote this toy example. fakeData[] will return 1-21 data points, and I want to run it until I have at least 100. But this code doesn't work because you can't take the so-far Length[] of a list that you're still building.
fakeData[n_] := RandomReal[1, 1 + RandomInteger[n]];
big = Reap[
While[Length[big] < 100, (* this doesn't work*)
Sow[fakeData[20]]]][[2, 1]]
I could just allocate 'big' as a Table with length 100 and copy each new small list into it, but then I'd have to discard some perfectly good data points I laboriously calculated, which is distasteful. Is there a better way?
sow-reap
asked Jul 23 at 18:01
Jerry GuernJerry Guern
2,11410 silver badges37 bronze badges
$endgroup$
I have a process that returns an unpredictable number of data points, and I'd like to run it repeatedly until I have a certain number of points.
My actual code is too complicated to use an illustration, so I wrote this toy example. fakeData[] will return 1-21 data points, and I want to run it until I have at least 100. But this code doesn't work because you can't take the so-far Length[] of a list that you're still building.
fakeData[n_] := RandomReal[1, 1 + RandomInteger[n]];
big = Reap[
While[Length[big] < 100, (* this doesn't work*)
Sow[fakeData[20]]]][[2, 1]]
I could just allocate 'big' as a Table with length 100 and copy each new small list into it, but then I'd have to discard some perfectly good data points I laboriously calculated, which is distasteful. Is there a better way?
sow-reap
sow-reap
asked Jul 23 at 18:01
Jerry GuernJerry Guern
2,11410 silver badges37 bronze badges
asked Jul 23 at 18:01
Jerry GuernJerry Guern
2,11410 silver badges37 bronze badges
asked Jul 23 at 18:01
Jerry GuernJerry Guern
2,11410 silver badges37 bronze badges
asked Jul 23 at 18:01
Jerry GuernJerry Guern
2,11410 silver badges37 bronze badges
asked Jul 23 at 18:01
Jerry GuernJerry Guern
2,11410 silver badges37 bronze badges
2,11410 silver badges37 bronze badges
$begingroup$
WillNestWhile[(Join[#, fakeData[20]]) &, , Length[#] < 100 &]work for you?
$endgroup$
– Shadowray
Jul 24 at 20:19
$begingroup$
@Shadowray That will work, but I've been told that using Join[] repeatedly is very, very slow because of all the recopying necessary every time you add a chunk.
$endgroup$
– Jerry Guern
Jul 25 at 0:05
add a comment
|
$begingroup$
WillNestWhile[(Join[#, fakeData[20]]) &, , Length[#] < 100 &]work for you?
$endgroup$
– Shadowray
Jul 24 at 20:19
$begingroup$
@Shadowray That will work, but I've been told that using Join[] repeatedly is very, very slow because of all the recopying necessary every time you add a chunk.
$endgroup$
– Jerry Guern
Jul 25 at 0:05
$begingroup$
Will
NestWhile[(Join[#, fakeData[20]]) &, , Length[#] < 100 &] work for you?$endgroup$
– Shadowray
Jul 24 at 20:19
$begingroup$
Will
NestWhile[(Join[#, fakeData[20]]) &, , Length[#] < 100 &] work for you?$endgroup$
– Shadowray
Jul 24 at 20:19
$begingroup$
@Shadowray That will work, but I've been told that using Join[] repeatedly is very, very slow because of all the recopying necessary every time you add a chunk.
$endgroup$
– Jerry Guern
Jul 25 at 0:05
$begingroup$
@Shadowray That will work, but I've been told that using Join[] repeatedly is very, very slow because of all the recopying necessary every time you add a chunk.
$endgroup$
– Jerry Guern
Jul 25 at 0:05
add a comment
|
3 Answers
3
active
oldest
votes
$begingroup$
How about:
SeedRandom[1]
Reap[NestWhile[Join[#, Sow@fakeData[20]] &, , LessThan[100]@*Length]][[2, 1]]
0.00683794, 0.0936818, 0.474619, 0.310422, 0.153631, 0.31649, 0.337261,
0.470877, 0.32728, 0.124887, 0.113682, 0.988692, 0.970078, 0.908979,
0.964289, 0.741987, 0.819242, 0.539713, 0.012502, 0.439595, 0.169709,
0.771071, 0.998221, 0.179295, 0.901812, 0.661701, 0.162254,
0.85584, 0.00132041, 0.784942, 0.693806, 0.687592, 0.525913, 0.842108,
0.203219, 0.495244, 0.909835, 0.464522, 0.115059, 0.443676, 0.712994,
0.439824, 0.245655, 0.562932, 0.370393, 0.934574, 0.550753, 0.136193,
0.390665, 0.941924, 0.743334, 0.296465, 0.114065, 0.612737, 0.596194,
0.32461, 0.713441, 0.225573, 0.387218, 0.55637, 0.336226, 0.90315, 0.333871,
0.188398, 0.129602, 0.265823, 0.750065, 0.757875, 0.679856, 0.0740267,
0.691003, 0.571181, 0.921954, 0.559011, 0.341209, 0.757399, 0.856246,
0.578542, 0.866321, 0.641392, 0.474307, 0.197374, 0.172371, 0.448029,
0.122614, 0.146429, 0.0648023, 0.514557, 0.320289, 0.510485, 0.00828315,
0.346533, 0.0588742, 0.436849, 0.305532, 0.767718, 0.254158, 0.345529,
0.208461, 0.315747, 0.367579, 0.521331, 0.36944, 0.566759
Another similar possibility:
SeedRandom[1]
Reap[NestWhile[Length @ Sow @ fakeData[20] &, 0, LessThan[100] @* Plus, All]][[2, 1]]
same answer
answered Jul 23 at 18:11
Carl WollCarl Woll
94.8k3 gold badges129 silver badges239 bronze badges
$endgroup$
$begingroup$
Okay, thank you, that seems to do what exactly I wanted, now I just have to study docs for a while to understand how/why it works. :-) May I ask, why did you put that SeedRandom[1] in there? I don't see it's purpose, but I assume you had expert-level reasons.
$endgroup$
– Jerry Guern
Jul 23 at 18:26
$begingroup$
BecausefakeDatacallsRandomRealandRandomIntegerand these random functions can have reproducible results if you specify the seed.
$endgroup$
– rhermans
Jul 23 at 18:28
$begingroup$
One issue with this is that it doubles the memory cost, no? With largeReaps that could be prohibitive
$endgroup$
– b3m2a1
Jul 23 at 20:15
add a comment
|
$begingroup$
The straight forward solution is to simply count the number of points you have sown, i.e.:
big = Module[
count = 0,
Reap[
While[ count < 100, count += Length@Sow[fakeData[20]] ]
][[2,1]]
]
answered Jul 23 at 18:16
sakrasakra
3,40814 silver badges30 bronze badges
$endgroup$
$begingroup$
Sorry, I misread your question. See updated answer.
$endgroup$
– sakra
Jul 23 at 18:31
1
$begingroup$
Oh, I see, you're right, I can just manually track the length as I as to it. Thanks.
$endgroup$
– Jerry Guern
Jul 23 at 22:26
add a comment
|
$begingroup$
Here's a method that just uses Bag since I think effectively that's what Reap and Sow are using. It's probably a bit slower than adding the lists directly and flattening after, but it's conceptually how you were thinking about the original problem:
bag = Internal`Bag[];
SeedRandom[1]
While[Internal`BagLength[bag] < 100,
Internal`StuffBag[bag, #] & /@ fakeData[20]
];
Internal`BagPart[bag, All]
0.00683794, 0.0936818, 0.474619, 0.310422, 0.153631, 0.31649, 0.337261,
0.470877, 0.32728, 0.124887, 0.113682, 0.988692, 0.970078, 0.908979,
0.964289, 0.741987, 0.819242, 0.539713, 0.012502, 0.439595, 0.169709,
0.771071, 0.998221, 0.179295, 0.901812, 0.661701, 0.162254, 0.85584,
0.00132041, 0.784942, 0.693806, 0.687592, 0.525913, 0.842108, 0.203219,
0.495244, 0.909835, 0.464522, 0.115059, 0.443676, 0.712994, 0.439824,
0.245655, 0.562932, 0.370393, 0.934574, 0.550753, 0.136193, 0.390665,
0.941924, 0.743334, 0.296465, 0.114065, 0.612737, 0.596194, 0.32461,
0.713441, 0.225573, 0.387218, 0.55637, 0.336226, 0.90315, 0.333871, 0.188398,
0.129602, 0.265823, 0.750065, 0.757875, 0.679856, 0.0740267, 0.691003,
0.571181, 0.921954, 0.559011, 0.341209, 0.757399, 0.856246, 0.578542,
0.866321, 0.641392, 0.474307, 0.197374, 0.172371, 0.448029, 0.122614,
0.146429, 0.0648023, 0.514557, 0.320289, 0.510485, 0.00828315, 0.346533,
0.0588742, 0.436849, 0.305532, 0.767718, 0.254158, 0.345529, 0.208461,
0.315747, 0.367579, 0.521331, 0.36944, 0.566759
answered Jul 23 at 20:19
b3m2a1b3m2a1
32.1k3 gold badges64 silver badges185 bronze badges
$endgroup$
$begingroup$
Oh, thank you. SE didn't notify me of this Answer for some reason, so I just saw it today. Yes, this is conceptually what I was trying to do, and I appreciate this glimpse into what MMA is doing 'under the hood'.
$endgroup$
– Jerry Guern
Oct 2 at 19:41
$begingroup$
I'm wondering though, if Bag is what Reap/Sow use internally, why would this be slower than adding lists, rather than much faster? I didn't understand that part of your Answer.
$endgroup$
– Jerry Guern
Oct 2 at 19:43
$begingroup$
@JerryGuern You’re making Mathematica loop a bunch of times rather than letting that happen at the C level whenFlattenis called. Basically I’m saying that Sakra’s answer will be the best in all likelihood. In general, you want to maintain a nestedListstructure when constructing your dataset because this is stored as a linked list. Next best is list of list since that will be pointers. Then when you need it cast to the proper array you’ll want to use.
$endgroup$
– b3m2a1
Oct 3 at 2:52
add a comment
|
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "387"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/4.0/"u003ecc by-sa 4.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmathematica.stackexchange.com%2fquestions%2f202621%2fhow-to-sow-until-ive-reapd-enough%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
How about:
SeedRandom[1]
Reap[NestWhile[Join[#, Sow@fakeData[20]] &, , LessThan[100]@*Length]][[2, 1]]
0.00683794, 0.0936818, 0.474619, 0.310422, 0.153631, 0.31649, 0.337261,
0.470877, 0.32728, 0.124887, 0.113682, 0.988692, 0.970078, 0.908979,
0.964289, 0.741987, 0.819242, 0.539713, 0.012502, 0.439595, 0.169709,
0.771071, 0.998221, 0.179295, 0.901812, 0.661701, 0.162254,
0.85584, 0.00132041, 0.784942, 0.693806, 0.687592, 0.525913, 0.842108,
0.203219, 0.495244, 0.909835, 0.464522, 0.115059, 0.443676, 0.712994,
0.439824, 0.245655, 0.562932, 0.370393, 0.934574, 0.550753, 0.136193,
0.390665, 0.941924, 0.743334, 0.296465, 0.114065, 0.612737, 0.596194,
0.32461, 0.713441, 0.225573, 0.387218, 0.55637, 0.336226, 0.90315, 0.333871,
0.188398, 0.129602, 0.265823, 0.750065, 0.757875, 0.679856, 0.0740267,
0.691003, 0.571181, 0.921954, 0.559011, 0.341209, 0.757399, 0.856246,
0.578542, 0.866321, 0.641392, 0.474307, 0.197374, 0.172371, 0.448029,
0.122614, 0.146429, 0.0648023, 0.514557, 0.320289, 0.510485, 0.00828315,
0.346533, 0.0588742, 0.436849, 0.305532, 0.767718, 0.254158, 0.345529,
0.208461, 0.315747, 0.367579, 0.521331, 0.36944, 0.566759
Another similar possibility:
SeedRandom[1]
Reap[NestWhile[Length @ Sow @ fakeData[20] &, 0, LessThan[100] @* Plus, All]][[2, 1]]
same answer
answered Jul 23 at 18:11
Carl WollCarl Woll
94.8k3 gold badges129 silver badges239 bronze badges
$endgroup$
$begingroup$
Okay, thank you, that seems to do what exactly I wanted, now I just have to study docs for a while to understand how/why it works. :-) May I ask, why did you put that SeedRandom[1] in there? I don't see it's purpose, but I assume you had expert-level reasons.
$endgroup$
– Jerry Guern
Jul 23 at 18:26
$begingroup$
BecausefakeDatacallsRandomRealandRandomIntegerand these random functions can have reproducible results if you specify the seed.
$endgroup$
– rhermans
Jul 23 at 18:28
$begingroup$
One issue with this is that it doubles the memory cost, no? With largeReaps that could be prohibitive
$endgroup$
– b3m2a1
Jul 23 at 20:15
add a comment
|
$begingroup$
How about:
SeedRandom[1]
Reap[NestWhile[Join[#, Sow@fakeData[20]] &, , LessThan[100]@*Length]][[2, 1]]
0.00683794, 0.0936818, 0.474619, 0.310422, 0.153631, 0.31649, 0.337261,
0.470877, 0.32728, 0.124887, 0.113682, 0.988692, 0.970078, 0.908979,
0.964289, 0.741987, 0.819242, 0.539713, 0.012502, 0.439595, 0.169709,
0.771071, 0.998221, 0.179295, 0.901812, 0.661701, 0.162254,
0.85584, 0.00132041, 0.784942, 0.693806, 0.687592, 0.525913, 0.842108,
0.203219, 0.495244, 0.909835, 0.464522, 0.115059, 0.443676, 0.712994,
0.439824, 0.245655, 0.562932, 0.370393, 0.934574, 0.550753, 0.136193,
0.390665, 0.941924, 0.743334, 0.296465, 0.114065, 0.612737, 0.596194,
0.32461, 0.713441, 0.225573, 0.387218, 0.55637, 0.336226, 0.90315, 0.333871,
0.188398, 0.129602, 0.265823, 0.750065, 0.757875, 0.679856, 0.0740267,
0.691003, 0.571181, 0.921954, 0.559011, 0.341209, 0.757399, 0.856246,
0.578542, 0.866321, 0.641392, 0.474307, 0.197374, 0.172371, 0.448029,
0.122614, 0.146429, 0.0648023, 0.514557, 0.320289, 0.510485, 0.00828315,
0.346533, 0.0588742, 0.436849, 0.305532, 0.767718, 0.254158, 0.345529,
0.208461, 0.315747, 0.367579, 0.521331, 0.36944, 0.566759
Another similar possibility:
SeedRandom[1]
Reap[NestWhile[Length @ Sow @ fakeData[20] &, 0, LessThan[100] @* Plus, All]][[2, 1]]
same answer
answered Jul 23 at 18:11
Carl WollCarl Woll
94.8k3 gold badges129 silver badges239 bronze badges
$endgroup$
$begingroup$
Okay, thank you, that seems to do what exactly I wanted, now I just have to study docs for a while to understand how/why it works. :-) May I ask, why did you put that SeedRandom[1] in there? I don't see it's purpose, but I assume you had expert-level reasons.
$endgroup$
– Jerry Guern
Jul 23 at 18:26
$begingroup$
BecausefakeDatacallsRandomRealandRandomIntegerand these random functions can have reproducible results if you specify the seed.
$endgroup$
– rhermans
Jul 23 at 18:28
$begingroup$
One issue with this is that it doubles the memory cost, no? With largeReaps that could be prohibitive
$endgroup$
– b3m2a1
Jul 23 at 20:15
add a comment
|
$begingroup$
How about:
SeedRandom[1]
Reap[NestWhile[Join[#, Sow@fakeData[20]] &, , LessThan[100]@*Length]][[2, 1]]
0.00683794, 0.0936818, 0.474619, 0.310422, 0.153631, 0.31649, 0.337261,
0.470877, 0.32728, 0.124887, 0.113682, 0.988692, 0.970078, 0.908979,
0.964289, 0.741987, 0.819242, 0.539713, 0.012502, 0.439595, 0.169709,
0.771071, 0.998221, 0.179295, 0.901812, 0.661701, 0.162254,
0.85584, 0.00132041, 0.784942, 0.693806, 0.687592, 0.525913, 0.842108,
0.203219, 0.495244, 0.909835, 0.464522, 0.115059, 0.443676, 0.712994,
0.439824, 0.245655, 0.562932, 0.370393, 0.934574, 0.550753, 0.136193,
0.390665, 0.941924, 0.743334, 0.296465, 0.114065, 0.612737, 0.596194,
0.32461, 0.713441, 0.225573, 0.387218, 0.55637, 0.336226, 0.90315, 0.333871,
0.188398, 0.129602, 0.265823, 0.750065, 0.757875, 0.679856, 0.0740267,
0.691003, 0.571181, 0.921954, 0.559011, 0.341209, 0.757399, 0.856246,
0.578542, 0.866321, 0.641392, 0.474307, 0.197374, 0.172371, 0.448029,
0.122614, 0.146429, 0.0648023, 0.514557, 0.320289, 0.510485, 0.00828315,
0.346533, 0.0588742, 0.436849, 0.305532, 0.767718, 0.254158, 0.345529,
0.208461, 0.315747, 0.367579, 0.521331, 0.36944, 0.566759
Another similar possibility:
SeedRandom[1]
Reap[NestWhile[Length @ Sow @ fakeData[20] &, 0, LessThan[100] @* Plus, All]][[2, 1]]
same answer
answered Jul 23 at 18:11
Carl WollCarl Woll
94.8k3 gold badges129 silver badges239 bronze badges
$endgroup$
How about:
SeedRandom[1]
Reap[NestWhile[Join[#, Sow@fakeData[20]] &, , LessThan[100]@*Length]][[2, 1]]
0.00683794, 0.0936818, 0.474619, 0.310422, 0.153631, 0.31649, 0.337261,
0.470877, 0.32728, 0.124887, 0.113682, 0.988692, 0.970078, 0.908979,
0.964289, 0.741987, 0.819242, 0.539713, 0.012502, 0.439595, 0.169709,
0.771071, 0.998221, 0.179295, 0.901812, 0.661701, 0.162254,
0.85584, 0.00132041, 0.784942, 0.693806, 0.687592, 0.525913, 0.842108,
0.203219, 0.495244, 0.909835, 0.464522, 0.115059, 0.443676, 0.712994,
0.439824, 0.245655, 0.562932, 0.370393, 0.934574, 0.550753, 0.136193,
0.390665, 0.941924, 0.743334, 0.296465, 0.114065, 0.612737, 0.596194,
0.32461, 0.713441, 0.225573, 0.387218, 0.55637, 0.336226, 0.90315, 0.333871,
0.188398, 0.129602, 0.265823, 0.750065, 0.757875, 0.679856, 0.0740267,
0.691003, 0.571181, 0.921954, 0.559011, 0.341209, 0.757399, 0.856246,
0.578542, 0.866321, 0.641392, 0.474307, 0.197374, 0.172371, 0.448029,
0.122614, 0.146429, 0.0648023, 0.514557, 0.320289, 0.510485, 0.00828315,
0.346533, 0.0588742, 0.436849, 0.305532, 0.767718, 0.254158, 0.345529,
0.208461, 0.315747, 0.367579, 0.521331, 0.36944, 0.566759
Another similar possibility:
SeedRandom[1]
Reap[NestWhile[Length @ Sow @ fakeData[20] &, 0, LessThan[100] @* Plus, All]][[2, 1]]
same answer
answered Jul 23 at 18:11
Carl WollCarl Woll
94.8k3 gold badges129 silver badges239 bronze badges
edited Jul 23 at 18:18
answered Jul 23 at 18:11
Carl WollCarl Woll
94.8k3 gold badges129 silver badges239 bronze badges
answered Jul 23 at 18:11
Carl WollCarl Woll
94.8k3 gold badges129 silver badges239 bronze badges
answered Jul 23 at 18:11
Carl WollCarl Woll
94.8k3 gold badges129 silver badges239 bronze badges
94.8k3 gold badges129 silver badges239 bronze badges
$begingroup$
Okay, thank you, that seems to do what exactly I wanted, now I just have to study docs for a while to understand how/why it works. :-) May I ask, why did you put that SeedRandom[1] in there? I don't see it's purpose, but I assume you had expert-level reasons.
$endgroup$
– Jerry Guern
Jul 23 at 18:26
$begingroup$
BecausefakeDatacallsRandomRealandRandomIntegerand these random functions can have reproducible results if you specify the seed.
$endgroup$
– rhermans
Jul 23 at 18:28
$begingroup$
One issue with this is that it doubles the memory cost, no? With largeReaps that could be prohibitive
$endgroup$
– b3m2a1
Jul 23 at 20:15
add a comment
|
$begingroup$
Okay, thank you, that seems to do what exactly I wanted, now I just have to study docs for a while to understand how/why it works. :-) May I ask, why did you put that SeedRandom[1] in there? I don't see it's purpose, but I assume you had expert-level reasons.
$endgroup$
– Jerry Guern
Jul 23 at 18:26
$begingroup$
BecausefakeDatacallsRandomRealandRandomIntegerand these random functions can have reproducible results if you specify the seed.
$endgroup$
– rhermans
Jul 23 at 18:28
$begingroup$
One issue with this is that it doubles the memory cost, no? With largeReaps that could be prohibitive
$endgroup$
– b3m2a1
Jul 23 at 20:15
$begingroup$
Okay, thank you, that seems to do what exactly I wanted, now I just have to study docs for a while to understand how/why it works. :-) May I ask, why did you put that SeedRandom[1] in there? I don't see it's purpose, but I assume you had expert-level reasons.
$endgroup$
– Jerry Guern
Jul 23 at 18:26
$begingroup$
Okay, thank you, that seems to do what exactly I wanted, now I just have to study docs for a while to understand how/why it works. :-) May I ask, why did you put that SeedRandom[1] in there? I don't see it's purpose, but I assume you had expert-level reasons.
$endgroup$
– Jerry Guern
Jul 23 at 18:26
$begingroup$
Because
fakeData calls RandomReal and RandomInteger and these random functions can have reproducible results if you specify the seed.$endgroup$
– rhermans
Jul 23 at 18:28
$begingroup$
Because
fakeData calls RandomReal and RandomInteger and these random functions can have reproducible results if you specify the seed.$endgroup$
– rhermans
Jul 23 at 18:28
$begingroup$
One issue with this is that it doubles the memory cost, no? With large
Reaps that could be prohibitive$endgroup$
– b3m2a1
Jul 23 at 20:15
$begingroup$
One issue with this is that it doubles the memory cost, no? With large
Reaps that could be prohibitive$endgroup$
– b3m2a1
Jul 23 at 20:15
add a comment
|
$begingroup$
The straight forward solution is to simply count the number of points you have sown, i.e.:
big = Module[
count = 0,
Reap[
While[ count < 100, count += Length@Sow[fakeData[20]] ]
][[2,1]]
]
answered Jul 23 at 18:16
sakrasakra
3,40814 silver badges30 bronze badges
$endgroup$
$begingroup$
Sorry, I misread your question. See updated answer.
$endgroup$
– sakra
Jul 23 at 18:31
1
$begingroup$
Oh, I see, you're right, I can just manually track the length as I as to it. Thanks.
$endgroup$
– Jerry Guern
Jul 23 at 22:26
add a comment
|
$begingroup$
The straight forward solution is to simply count the number of points you have sown, i.e.:
big = Module[
count = 0,
Reap[
While[ count < 100, count += Length@Sow[fakeData[20]] ]
][[2,1]]
]
answered Jul 23 at 18:16
sakrasakra
3,40814 silver badges30 bronze badges
$endgroup$
$begingroup$
Sorry, I misread your question. See updated answer.
$endgroup$
– sakra
Jul 23 at 18:31
1
$begingroup$
Oh, I see, you're right, I can just manually track the length as I as to it. Thanks.
$endgroup$
– Jerry Guern
Jul 23 at 22:26
add a comment
|
$begingroup$
The straight forward solution is to simply count the number of points you have sown, i.e.:
big = Module[
count = 0,
Reap[
While[ count < 100, count += Length@Sow[fakeData[20]] ]
][[2,1]]
]
answered Jul 23 at 18:16
sakrasakra
3,40814 silver badges30 bronze badges
$endgroup$
The straight forward solution is to simply count the number of points you have sown, i.e.:
big = Module[
count = 0,
Reap[
While[ count < 100, count += Length@Sow[fakeData[20]] ]
][[2,1]]
]
answered Jul 23 at 18:16
sakrasakra
3,40814 silver badges30 bronze badges
edited Jul 23 at 18:31
answered Jul 23 at 18:16
sakrasakra
3,40814 silver badges30 bronze badges
answered Jul 23 at 18:16
sakrasakra
3,40814 silver badges30 bronze badges
answered Jul 23 at 18:16
sakrasakra
3,40814 silver badges30 bronze badges
3,40814 silver badges30 bronze badges
$begingroup$
Sorry, I misread your question. See updated answer.
$endgroup$
– sakra
Jul 23 at 18:31
1
$begingroup$
Oh, I see, you're right, I can just manually track the length as I as to it. Thanks.
$endgroup$
– Jerry Guern
Jul 23 at 22:26
add a comment
|
$begingroup$
Sorry, I misread your question. See updated answer.
$endgroup$
– sakra
Jul 23 at 18:31
1
$begingroup$
Oh, I see, you're right, I can just manually track the length as I as to it. Thanks.
$endgroup$
– Jerry Guern
Jul 23 at 22:26
$begingroup$
Sorry, I misread your question. See updated answer.
$endgroup$
– sakra
Jul 23 at 18:31
$begingroup$
Sorry, I misread your question. See updated answer.
$endgroup$
– sakra
Jul 23 at 18:31
1
1
$begingroup$
Oh, I see, you're right, I can just manually track the length as I as to it. Thanks.
$endgroup$
– Jerry Guern
Jul 23 at 22:26
$begingroup$
Oh, I see, you're right, I can just manually track the length as I as to it. Thanks.
$endgroup$
– Jerry Guern
Jul 23 at 22:26
add a comment
|
$begingroup$
Here's a method that just uses Bag since I think effectively that's what Reap and Sow are using. It's probably a bit slower than adding the lists directly and flattening after, but it's conceptually how you were thinking about the original problem:
bag = Internal`Bag[];
SeedRandom[1]
While[Internal`BagLength[bag] < 100,
Internal`StuffBag[bag, #] & /@ fakeData[20]
];
Internal`BagPart[bag, All]
0.00683794, 0.0936818, 0.474619, 0.310422, 0.153631, 0.31649, 0.337261,
0.470877, 0.32728, 0.124887, 0.113682, 0.988692, 0.970078, 0.908979,
0.964289, 0.741987, 0.819242, 0.539713, 0.012502, 0.439595, 0.169709,
0.771071, 0.998221, 0.179295, 0.901812, 0.661701, 0.162254, 0.85584,
0.00132041, 0.784942, 0.693806, 0.687592, 0.525913, 0.842108, 0.203219,
0.495244, 0.909835, 0.464522, 0.115059, 0.443676, 0.712994, 0.439824,
0.245655, 0.562932, 0.370393, 0.934574, 0.550753, 0.136193, 0.390665,
0.941924, 0.743334, 0.296465, 0.114065, 0.612737, 0.596194, 0.32461,
0.713441, 0.225573, 0.387218, 0.55637, 0.336226, 0.90315, 0.333871, 0.188398,
0.129602, 0.265823, 0.750065, 0.757875, 0.679856, 0.0740267, 0.691003,
0.571181, 0.921954, 0.559011, 0.341209, 0.757399, 0.856246, 0.578542,
0.866321, 0.641392, 0.474307, 0.197374, 0.172371, 0.448029, 0.122614,
0.146429, 0.0648023, 0.514557, 0.320289, 0.510485, 0.00828315, 0.346533,
0.0588742, 0.436849, 0.305532, 0.767718, 0.254158, 0.345529, 0.208461,
0.315747, 0.367579, 0.521331, 0.36944, 0.566759
answered Jul 23 at 20:19
b3m2a1b3m2a1
32.1k3 gold badges64 silver badges185 bronze badges
$endgroup$
$begingroup$
Oh, thank you. SE didn't notify me of this Answer for some reason, so I just saw it today. Yes, this is conceptually what I was trying to do, and I appreciate this glimpse into what MMA is doing 'under the hood'.
$endgroup$
– Jerry Guern
Oct 2 at 19:41
$begingroup$
I'm wondering though, if Bag is what Reap/Sow use internally, why would this be slower than adding lists, rather than much faster? I didn't understand that part of your Answer.
$endgroup$
– Jerry Guern
Oct 2 at 19:43
$begingroup$
@JerryGuern You’re making Mathematica loop a bunch of times rather than letting that happen at the C level whenFlattenis called. Basically I’m saying that Sakra’s answer will be the best in all likelihood. In general, you want to maintain a nestedListstructure when constructing your dataset because this is stored as a linked list. Next best is list of list since that will be pointers. Then when you need it cast to the proper array you’ll want to use.
$endgroup$
– b3m2a1
Oct 3 at 2:52
add a comment
|
$begingroup$
Here's a method that just uses Bag since I think effectively that's what Reap and Sow are using. It's probably a bit slower than adding the lists directly and flattening after, but it's conceptually how you were thinking about the original problem:
bag = Internal`Bag[];
SeedRandom[1]
While[Internal`BagLength[bag] < 100,
Internal`StuffBag[bag, #] & /@ fakeData[20]
];
Internal`BagPart[bag, All]
0.00683794, 0.0936818, 0.474619, 0.310422, 0.153631, 0.31649, 0.337261,
0.470877, 0.32728, 0.124887, 0.113682, 0.988692, 0.970078, 0.908979,
0.964289, 0.741987, 0.819242, 0.539713, 0.012502, 0.439595, 0.169709,
0.771071, 0.998221, 0.179295, 0.901812, 0.661701, 0.162254, 0.85584,
0.00132041, 0.784942, 0.693806, 0.687592, 0.525913, 0.842108, 0.203219,
0.495244, 0.909835, 0.464522, 0.115059, 0.443676, 0.712994, 0.439824,
0.245655, 0.562932, 0.370393, 0.934574, 0.550753, 0.136193, 0.390665,
0.941924, 0.743334, 0.296465, 0.114065, 0.612737, 0.596194, 0.32461,
0.713441, 0.225573, 0.387218, 0.55637, 0.336226, 0.90315, 0.333871, 0.188398,
0.129602, 0.265823, 0.750065, 0.757875, 0.679856, 0.0740267, 0.691003,
0.571181, 0.921954, 0.559011, 0.341209, 0.757399, 0.856246, 0.578542,
0.866321, 0.641392, 0.474307, 0.197374, 0.172371, 0.448029, 0.122614,
0.146429, 0.0648023, 0.514557, 0.320289, 0.510485, 0.00828315, 0.346533,
0.0588742, 0.436849, 0.305532, 0.767718, 0.254158, 0.345529, 0.208461,
0.315747, 0.367579, 0.521331, 0.36944, 0.566759
answered Jul 23 at 20:19
b3m2a1b3m2a1
32.1k3 gold badges64 silver badges185 bronze badges
$endgroup$
$begingroup$
Oh, thank you. SE didn't notify me of this Answer for some reason, so I just saw it today. Yes, this is conceptually what I was trying to do, and I appreciate this glimpse into what MMA is doing 'under the hood'.
$endgroup$
– Jerry Guern
Oct 2 at 19:41
$begingroup$
I'm wondering though, if Bag is what Reap/Sow use internally, why would this be slower than adding lists, rather than much faster? I didn't understand that part of your Answer.
$endgroup$
– Jerry Guern
Oct 2 at 19:43
$begingroup$
@JerryGuern You’re making Mathematica loop a bunch of times rather than letting that happen at the C level whenFlattenis called. Basically I’m saying that Sakra’s answer will be the best in all likelihood. In general, you want to maintain a nestedListstructure when constructing your dataset because this is stored as a linked list. Next best is list of list since that will be pointers. Then when you need it cast to the proper array you’ll want to use.
$endgroup$
– b3m2a1
Oct 3 at 2:52
add a comment
|
$begingroup$
Here's a method that just uses Bag since I think effectively that's what Reap and Sow are using. It's probably a bit slower than adding the lists directly and flattening after, but it's conceptually how you were thinking about the original problem:
bag = Internal`Bag[];
SeedRandom[1]
While[Internal`BagLength[bag] < 100,
Internal`StuffBag[bag, #] & /@ fakeData[20]
];
Internal`BagPart[bag, All]
0.00683794, 0.0936818, 0.474619, 0.310422, 0.153631, 0.31649, 0.337261,
0.470877, 0.32728, 0.124887, 0.113682, 0.988692, 0.970078, 0.908979,
0.964289, 0.741987, 0.819242, 0.539713, 0.012502, 0.439595, 0.169709,
0.771071, 0.998221, 0.179295, 0.901812, 0.661701, 0.162254, 0.85584,
0.00132041, 0.784942, 0.693806, 0.687592, 0.525913, 0.842108, 0.203219,
0.495244, 0.909835, 0.464522, 0.115059, 0.443676, 0.712994, 0.439824,
0.245655, 0.562932, 0.370393, 0.934574, 0.550753, 0.136193, 0.390665,
0.941924, 0.743334, 0.296465, 0.114065, 0.612737, 0.596194, 0.32461,
0.713441, 0.225573, 0.387218, 0.55637, 0.336226, 0.90315, 0.333871, 0.188398,
0.129602, 0.265823, 0.750065, 0.757875, 0.679856, 0.0740267, 0.691003,
0.571181, 0.921954, 0.559011, 0.341209, 0.757399, 0.856246, 0.578542,
0.866321, 0.641392, 0.474307, 0.197374, 0.172371, 0.448029, 0.122614,
0.146429, 0.0648023, 0.514557, 0.320289, 0.510485, 0.00828315, 0.346533,
0.0588742, 0.436849, 0.305532, 0.767718, 0.254158, 0.345529, 0.208461,
0.315747, 0.367579, 0.521331, 0.36944, 0.566759
answered Jul 23 at 20:19
b3m2a1b3m2a1
32.1k3 gold badges64 silver badges185 bronze badges
$endgroup$
Here's a method that just uses Bag since I think effectively that's what Reap and Sow are using. It's probably a bit slower than adding the lists directly and flattening after, but it's conceptually how you were thinking about the original problem:
bag = Internal`Bag[];
SeedRandom[1]
While[Internal`BagLength[bag] < 100,
Internal`StuffBag[bag, #] & /@ fakeData[20]
];
Internal`BagPart[bag, All]
0.00683794, 0.0936818, 0.474619, 0.310422, 0.153631, 0.31649, 0.337261,
0.470877, 0.32728, 0.124887, 0.113682, 0.988692, 0.970078, 0.908979,
0.964289, 0.741987, 0.819242, 0.539713, 0.012502, 0.439595, 0.169709,
0.771071, 0.998221, 0.179295, 0.901812, 0.661701, 0.162254, 0.85584,
0.00132041, 0.784942, 0.693806, 0.687592, 0.525913, 0.842108, 0.203219,
0.495244, 0.909835, 0.464522, 0.115059, 0.443676, 0.712994, 0.439824,
0.245655, 0.562932, 0.370393, 0.934574, 0.550753, 0.136193, 0.390665,
0.941924, 0.743334, 0.296465, 0.114065, 0.612737, 0.596194, 0.32461,
0.713441, 0.225573, 0.387218, 0.55637, 0.336226, 0.90315, 0.333871, 0.188398,
0.129602, 0.265823, 0.750065, 0.757875, 0.679856, 0.0740267, 0.691003,
0.571181, 0.921954, 0.559011, 0.341209, 0.757399, 0.856246, 0.578542,
0.866321, 0.641392, 0.474307, 0.197374, 0.172371, 0.448029, 0.122614,
0.146429, 0.0648023, 0.514557, 0.320289, 0.510485, 0.00828315, 0.346533,
0.0588742, 0.436849, 0.305532, 0.767718, 0.254158, 0.345529, 0.208461,
0.315747, 0.367579, 0.521331, 0.36944, 0.566759
answered Jul 23 at 20:19
b3m2a1b3m2a1
32.1k3 gold badges64 silver badges185 bronze badges
answered Jul 23 at 20:19
b3m2a1b3m2a1
32.1k3 gold badges64 silver badges185 bronze badges
answered Jul 23 at 20:19
b3m2a1b3m2a1
32.1k3 gold badges64 silver badges185 bronze badges
answered Jul 23 at 20:19
b3m2a1b3m2a1
32.1k3 gold badges64 silver badges185 bronze badges
32.1k3 gold badges64 silver badges185 bronze badges
$begingroup$
Oh, thank you. SE didn't notify me of this Answer for some reason, so I just saw it today. Yes, this is conceptually what I was trying to do, and I appreciate this glimpse into what MMA is doing 'under the hood'.
$endgroup$
– Jerry Guern
Oct 2 at 19:41
$begingroup$
I'm wondering though, if Bag is what Reap/Sow use internally, why would this be slower than adding lists, rather than much faster? I didn't understand that part of your Answer.
$endgroup$
– Jerry Guern
Oct 2 at 19:43
$begingroup$
@JerryGuern You’re making Mathematica loop a bunch of times rather than letting that happen at the C level whenFlattenis called. Basically I’m saying that Sakra’s answer will be the best in all likelihood. In general, you want to maintain a nestedListstructure when constructing your dataset because this is stored as a linked list. Next best is list of list since that will be pointers. Then when you need it cast to the proper array you’ll want to use.
$endgroup$
– b3m2a1
Oct 3 at 2:52
add a comment
|
$begingroup$
Oh, thank you. SE didn't notify me of this Answer for some reason, so I just saw it today. Yes, this is conceptually what I was trying to do, and I appreciate this glimpse into what MMA is doing 'under the hood'.
$endgroup$
– Jerry Guern
Oct 2 at 19:41
$begingroup$
I'm wondering though, if Bag is what Reap/Sow use internally, why would this be slower than adding lists, rather than much faster? I didn't understand that part of your Answer.
$endgroup$
– Jerry Guern
Oct 2 at 19:43
$begingroup$
@JerryGuern You’re making Mathematica loop a bunch of times rather than letting that happen at the C level whenFlattenis called. Basically I’m saying that Sakra’s answer will be the best in all likelihood. In general, you want to maintain a nestedListstructure when constructing your dataset because this is stored as a linked list. Next best is list of list since that will be pointers. Then when you need it cast to the proper array you’ll want to use.
$endgroup$
– b3m2a1
Oct 3 at 2:52
$begingroup$
Oh, thank you. SE didn't notify me of this Answer for some reason, so I just saw it today. Yes, this is conceptually what I was trying to do, and I appreciate this glimpse into what MMA is doing 'under the hood'.
$endgroup$
– Jerry Guern
Oct 2 at 19:41
$begingroup$
Oh, thank you. SE didn't notify me of this Answer for some reason, so I just saw it today. Yes, this is conceptually what I was trying to do, and I appreciate this glimpse into what MMA is doing 'under the hood'.
$endgroup$
– Jerry Guern
Oct 2 at 19:41
$begingroup$
I'm wondering though, if Bag is what Reap/Sow use internally, why would this be slower than adding lists, rather than much faster? I didn't understand that part of your Answer.
$endgroup$
– Jerry Guern
Oct 2 at 19:43
$begingroup$
I'm wondering though, if Bag is what Reap/Sow use internally, why would this be slower than adding lists, rather than much faster? I didn't understand that part of your Answer.
$endgroup$
– Jerry Guern
Oct 2 at 19:43
$begingroup$
@JerryGuern You’re making Mathematica loop a bunch of times rather than letting that happen at the C level when
Flatten is called. Basically I’m saying that Sakra’s answer will be the best in all likelihood. In general, you want to maintain a nested List structure when constructing your dataset because this is stored as a linked list. Next best is list of list since that will be pointers. Then when you need it cast to the proper array you’ll want to use.$endgroup$
– b3m2a1
Oct 3 at 2:52
$begingroup$
@JerryGuern You’re making Mathematica loop a bunch of times rather than letting that happen at the C level when
Flatten is called. Basically I’m saying that Sakra’s answer will be the best in all likelihood. In general, you want to maintain a nested List structure when constructing your dataset because this is stored as a linked list. Next best is list of list since that will be pointers. Then when you need it cast to the proper array you’ll want to use.$endgroup$
– b3m2a1
Oct 3 at 2:52
add a comment
|
Thanks for contributing an answer to Mathematica Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmathematica.stackexchange.com%2fquestions%2f202621%2fhow-to-sow-until-ive-reapd-enough%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Will
NestWhile[(Join[#, fakeData[20]]) &, , Length[#] < 100 &]work for you?$endgroup$
– Shadowray
Jul 24 at 20:19
$begingroup$
@Shadowray That will work, but I've been told that using Join[] repeatedly is very, very slow because of all the recopying necessary every time you add a chunk.
$endgroup$
– Jerry Guern
Jul 25 at 0:05