Optimising Table wrapping over a SelectWVM hangs or crashes on empty array when compiled ListableGetting lengths of sublists that sum to more than oneIssue with very large lists in MathematicaFinding planetary conjunctions with Mathematica (project-level)Select on 3D-table along diagonalLooping with “Table” over two variablesUsing Apply over multi-dimensional tableParallelTable[ParallelTable] vs Table[ParallelTable]: inexplicable differences & nonlinear scaling in computation timeHow can I implement a repeating operation?Creating a dynamic list with different subset lengths in while loopMaking polynomials representing frequency of a character in a list

How can an advanced civilization forget how to manufacture its technology?

<schwitz>, <zwinker> etc. Does German always use 2nd Person Singular Imperative verbs for emoticons? If so, why?

Why is dry soil hydrophobic? Bad gardener paradox

How does Fury know about this in Spider-Man: Far From Home?

Stuck Apple Mail - how to reset?

Referring to different instances of the same character in time travel

What's the minimum number of sensors for a hobby GPS waypoint-following UAV?

Are neural networks prone to catastrophic forgetting?

I quit, and boss offered me 3 month "grace period" where I could still come back

When did the Roman Empire fall according to contemporaries?

How can I deal with a player trying to insert real-world mythology into my homebrew setting?

How can I get both Giga Drain and Mach Punch on Breloom?

Redirect https to fqdn

Why would guns not work in the dungeon?

Would letting a multiclass character rebuild their character to be single-classed be game-breaking?

Dropping outliers based on "2.5 times the RMSE"

What is this welding tool I found in my attic?

Which states have a head of state or government from another country?

How to check the quality of an audio sample?

Shortest distance around a pyramid

Is a Lisp program in both prog-mode and lisp-mode?

I have a ruthless DM and I'm considering leaving the party. What are my options to minimize the negative impact to the rest of the group?

How the name "craqueuhhe" is read

How to know whether a Tamron lens is compatible with Canon EOS 60D?



Optimising Table wrapping over a Select


WVM hangs or crashes on empty array when compiled ListableGetting lengths of sublists that sum to more than oneIssue with very large lists in MathematicaFinding planetary conjunctions with Mathematica (project-level)Select on 3D-table along diagonalLooping with “Table” over two variablesUsing Apply over multi-dimensional tableParallelTable[ParallelTable] vs Table[ParallelTable]: inexplicable differences & nonlinear scaling in computation timeHow can I implement a repeating operation?Creating a dynamic list with different subset lengths in while loopMaking polynomials representing frequency of a character in a list






.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;








6












$begingroup$


Suppose we have the following lists:



L0="a", "a", "h", "a", "d", "k", "r", "v", "a", "b", "c", 
"k", "a", "b", "c", "s", "u", "a", "b", "f", "t", "a", "b",
"e", "l", "n", "o", "a", "b", "d", "n", "o", "a", "b", "d", "e",
"n", "o", "a", "b", "d", "e", "m", "n", "o", "t";
L1="a", "b", "a", "c", "a", "d", "a", "e", "a", "f";


The aim is to go through L0 for each pair in L1 and count how many times the given pair appears in L0 and categorise them into a polynomial with respect to lengths. I do as follow:



Table[Total[ 
x^Map[Length, Select[L0, SubsetQ[ToLowerCase[#1], L1[[i]]] &]]], i,
Length[L1]]


which gives:



2 x^4 + 2 x^5 + 2 x^6 + x^8, x^4 + x^5, 2 x^5 + x^6 + x^8, 
2 x^6 + x^8, x^4


So to explain more clearly the first pair is "a","b", and we see that "a","b" appears in elements that have length 4 in L0 two times, two times in elements of size 5 and so on. I wonder how can I make this Table operation faster. This is a sample I'm showing here the main dataset has thousands of elements and there are thousands of curves.










share|improve this question











$endgroup$











  • $begingroup$
    I believe you've made a typo here: "we see that "a","b" appears in elements that have length 2 in L0 two times". Did you mean to say in elements that have length 4?
    $endgroup$
    – amator2357
    8 hours ago










  • $begingroup$
    Oh yes that's a typo :) thanks
    $endgroup$
    – William
    7 hours ago










  • $begingroup$
    Not a problem :)
    $endgroup$
    – amator2357
    7 hours ago










  • $begingroup$
    Does L1 always consist of list of same length? Or is the Length/@ L1 not constant?
    $endgroup$
    – Henrik Schumacher
    7 hours ago










  • $begingroup$
    @HenrikSchumacher elements of L1 are always have length =2
    $endgroup$
    – William
    7 hours ago

















6












$begingroup$


Suppose we have the following lists:



L0="a", "a", "h", "a", "d", "k", "r", "v", "a", "b", "c", 
"k", "a", "b", "c", "s", "u", "a", "b", "f", "t", "a", "b",
"e", "l", "n", "o", "a", "b", "d", "n", "o", "a", "b", "d", "e",
"n", "o", "a", "b", "d", "e", "m", "n", "o", "t";
L1="a", "b", "a", "c", "a", "d", "a", "e", "a", "f";


The aim is to go through L0 for each pair in L1 and count how many times the given pair appears in L0 and categorise them into a polynomial with respect to lengths. I do as follow:



Table[Total[ 
x^Map[Length, Select[L0, SubsetQ[ToLowerCase[#1], L1[[i]]] &]]], i,
Length[L1]]


which gives:



2 x^4 + 2 x^5 + 2 x^6 + x^8, x^4 + x^5, 2 x^5 + x^6 + x^8, 
2 x^6 + x^8, x^4


So to explain more clearly the first pair is "a","b", and we see that "a","b" appears in elements that have length 4 in L0 two times, two times in elements of size 5 and so on. I wonder how can I make this Table operation faster. This is a sample I'm showing here the main dataset has thousands of elements and there are thousands of curves.










share|improve this question











$endgroup$











  • $begingroup$
    I believe you've made a typo here: "we see that "a","b" appears in elements that have length 2 in L0 two times". Did you mean to say in elements that have length 4?
    $endgroup$
    – amator2357
    8 hours ago










  • $begingroup$
    Oh yes that's a typo :) thanks
    $endgroup$
    – William
    7 hours ago










  • $begingroup$
    Not a problem :)
    $endgroup$
    – amator2357
    7 hours ago










  • $begingroup$
    Does L1 always consist of list of same length? Or is the Length/@ L1 not constant?
    $endgroup$
    – Henrik Schumacher
    7 hours ago










  • $begingroup$
    @HenrikSchumacher elements of L1 are always have length =2
    $endgroup$
    – William
    7 hours ago













6












6








6





$begingroup$


Suppose we have the following lists:



L0="a", "a", "h", "a", "d", "k", "r", "v", "a", "b", "c", 
"k", "a", "b", "c", "s", "u", "a", "b", "f", "t", "a", "b",
"e", "l", "n", "o", "a", "b", "d", "n", "o", "a", "b", "d", "e",
"n", "o", "a", "b", "d", "e", "m", "n", "o", "t";
L1="a", "b", "a", "c", "a", "d", "a", "e", "a", "f";


The aim is to go through L0 for each pair in L1 and count how many times the given pair appears in L0 and categorise them into a polynomial with respect to lengths. I do as follow:



Table[Total[ 
x^Map[Length, Select[L0, SubsetQ[ToLowerCase[#1], L1[[i]]] &]]], i,
Length[L1]]


which gives:



2 x^4 + 2 x^5 + 2 x^6 + x^8, x^4 + x^5, 2 x^5 + x^6 + x^8, 
2 x^6 + x^8, x^4


So to explain more clearly the first pair is "a","b", and we see that "a","b" appears in elements that have length 4 in L0 two times, two times in elements of size 5 and so on. I wonder how can I make this Table operation faster. This is a sample I'm showing here the main dataset has thousands of elements and there are thousands of curves.










share|improve this question











$endgroup$




Suppose we have the following lists:



L0="a", "a", "h", "a", "d", "k", "r", "v", "a", "b", "c", 
"k", "a", "b", "c", "s", "u", "a", "b", "f", "t", "a", "b",
"e", "l", "n", "o", "a", "b", "d", "n", "o", "a", "b", "d", "e",
"n", "o", "a", "b", "d", "e", "m", "n", "o", "t";
L1="a", "b", "a", "c", "a", "d", "a", "e", "a", "f";


The aim is to go through L0 for each pair in L1 and count how many times the given pair appears in L0 and categorise them into a polynomial with respect to lengths. I do as follow:



Table[Total[ 
x^Map[Length, Select[L0, SubsetQ[ToLowerCase[#1], L1[[i]]] &]]], i,
Length[L1]]


which gives:



2 x^4 + 2 x^5 + 2 x^6 + x^8, x^4 + x^5, 2 x^5 + x^6 + x^8, 
2 x^6 + x^8, x^4


So to explain more clearly the first pair is "a","b", and we see that "a","b" appears in elements that have length 4 in L0 two times, two times in elements of size 5 and so on. I wonder how can I make this Table operation faster. This is a sample I'm showing here the main dataset has thousands of elements and there are thousands of curves.







list-manipulation performance-tuning table filtering






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited 7 hours ago







William

















asked 8 hours ago









WilliamWilliam

9445 silver badges8 bronze badges




9445 silver badges8 bronze badges











  • $begingroup$
    I believe you've made a typo here: "we see that "a","b" appears in elements that have length 2 in L0 two times". Did you mean to say in elements that have length 4?
    $endgroup$
    – amator2357
    8 hours ago










  • $begingroup$
    Oh yes that's a typo :) thanks
    $endgroup$
    – William
    7 hours ago










  • $begingroup$
    Not a problem :)
    $endgroup$
    – amator2357
    7 hours ago










  • $begingroup$
    Does L1 always consist of list of same length? Or is the Length/@ L1 not constant?
    $endgroup$
    – Henrik Schumacher
    7 hours ago










  • $begingroup$
    @HenrikSchumacher elements of L1 are always have length =2
    $endgroup$
    – William
    7 hours ago
















  • $begingroup$
    I believe you've made a typo here: "we see that "a","b" appears in elements that have length 2 in L0 two times". Did you mean to say in elements that have length 4?
    $endgroup$
    – amator2357
    8 hours ago










  • $begingroup$
    Oh yes that's a typo :) thanks
    $endgroup$
    – William
    7 hours ago










  • $begingroup$
    Not a problem :)
    $endgroup$
    – amator2357
    7 hours ago










  • $begingroup$
    Does L1 always consist of list of same length? Or is the Length/@ L1 not constant?
    $endgroup$
    – Henrik Schumacher
    7 hours ago










  • $begingroup$
    @HenrikSchumacher elements of L1 are always have length =2
    $endgroup$
    – William
    7 hours ago















$begingroup$
I believe you've made a typo here: "we see that "a","b" appears in elements that have length 2 in L0 two times". Did you mean to say in elements that have length 4?
$endgroup$
– amator2357
8 hours ago




$begingroup$
I believe you've made a typo here: "we see that "a","b" appears in elements that have length 2 in L0 two times". Did you mean to say in elements that have length 4?
$endgroup$
– amator2357
8 hours ago












$begingroup$
Oh yes that's a typo :) thanks
$endgroup$
– William
7 hours ago




$begingroup$
Oh yes that's a typo :) thanks
$endgroup$
– William
7 hours ago












$begingroup$
Not a problem :)
$endgroup$
– amator2357
7 hours ago




$begingroup$
Not a problem :)
$endgroup$
– amator2357
7 hours ago












$begingroup$
Does L1 always consist of list of same length? Or is the Length/@ L1 not constant?
$endgroup$
– Henrik Schumacher
7 hours ago




$begingroup$
Does L1 always consist of list of same length? Or is the Length/@ L1 not constant?
$endgroup$
– Henrik Schumacher
7 hours ago












$begingroup$
@HenrikSchumacher elements of L1 are always have length =2
$endgroup$
– William
7 hours ago




$begingroup$
@HenrikSchumacher elements of L1 are always have length =2
$endgroup$
– William
7 hours ago










4 Answers
4






active

oldest

votes


















4












$begingroup$

Map[Total[x^Cases[L0, p:OrderlessPatternSequence[## & @@ #, ___] :> Length[p], All]] &]@L1



2 x^4 + 2 x^5 + 2 x^6 + x^8, x^4 + x^5, 2 x^5 + x^6 + x^8,
2 x^6 + x^8, x^4







share|improve this answer









$endgroup$




















    3












    $begingroup$

    To start off with, this has a 5-fold speed improvement on my machine. First, one helper function:



    findIntersectionsByLength[a_, l_] := 
    Map[If[Intersection[a, #] == a, Length[#], 0] &, l];


    This takes a set a and searches for all members of l for which a intersect l[[i]] is a. If there is a match, then it returns the length of the match, otherwise it returns 0, for each member of l.



    Then we tally these intersections over L0, delete all of the 0 returns, replace them with the polynomial form, and total them, for each element of L1. The Sort is included because Intersection sorts its results as well, so this ensures that it won't fail because "a","b" != "b","a".



    Table[Total[
    DeleteCases[
    Tally[findIntersectionsByLength[Sort[l], L0]], 0, _] /. e_?NumericQ,
    n_?NumericQ :> n x^e], l, L1]


    To test this for larger samples, I generated a large sample L0 as follows:



    alphabet = 
    FromCharacterCode[
    List /@ ToCharacterCode["abcdefghijklmnopqrstuvwxyz"]];
    L0 = Table[
    RandomSample[alphabet, RandomInteger[1, Length[Alphabet]]], i,
    1, 1000000];


    Using the same L1 as in the question, I get the following timings:



    AbsoluteTiming[
    res1 = Table[
    Total[x^Map[Length,
    Select[L0, SubsetQ[ToLowerCase[#1], L1[[i]]] &]]], i,
    Length[L1]];]



    34.0179, Null




    AbsoluteTiming[
    res2 = Table[
    Total[DeleteCases[
    Tally[findIntersectionsByLength[Sort[l], L0]], 0, _] /. e_?
    NumericQ, n_?NumericQ :> n x^e], l, L1];]



    6.74027, Null




    res1 === res2



    True







    share|improve this answer









    $endgroup$




















      3












      $begingroup$

      A bit ugly, but also works:



      Dot[Power[x,First[#1]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___]),#[[2]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___])]& /@ L1


      And seems to be quite quick. For the large sample @eyorble generated I get:



      AbsoluteTiming[Dot[Power[x,First[#1]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___]),#[[2]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___])]& /@ L1]



      0.623584, 0, 0, 0, 0, 0




      And for @eyorble's algorithm:




      5.83138, 0, 0, 0, 0, 0




      For your example we get:




      2 x^4 + 2 x^5 + 2 x^6 + x^8, x^4 + x^5, 2 x^5 + x^6 + x^8,
      2 x^6 + x^8, x^4




      , as required.






      share|improve this answer









      $endgroup$




















        2












        $begingroup$

        I'd like to employ SparseArrayand thus, I convert the characters to integers in Range[1,26]. Actually, I start with integers and convert them to characters (just to be able to test against the original implementation).



        n = 10000;
        m = 100;
        SeedRandom[1234];
        LL0 = Table[RandomChoice[1 ;; 26, RandomInteger[1, 26]], m];
        LL1 = RandomChoice[1 ;; 26, n, 2];
        L0 = (FromCharacterCode[Partition[#, 1]]) & /@ (LL0 + 96);
        L1 = (FromCharacterCode[Partition[#, 1]]) & /@ (LL1 + 96);


        OP's implementation



        First@AbsoluteTiming[
        result1 =
        Table[Total[
        x^Map[Length,
        Select[L0, SubsetQ[ToLowerCase[#1], L1[[i]]] &]]], i,
        Length[L1]];
        ]



        11.3748




        eyorblade's implementation:



        findIntersectionsByLength[a_, l_] := Map[If[Intersection[a, #] == a, Length[#], 0] &, l];

        result2 =
        Table[Total[
        DeleteCases[
        Tally[findIntersectionsByLength[Sort[l], L0]], 0, _] /. e_?
        NumericQ, n_?NumericQ :> n x^e], l, L1]; // AbsoluteTiming



        2.9406




        My implementation:



        cf = Compile[len, _Integer, 1, idx, _Integer, 1,
        If[Length[idx] == 1,
        Most[0],
        Part[len, Most[idx]]
        ],
        CompilationTarget -> "C",
        RuntimeAttributes -> Listable,
        Parallelization -> True,
        RuntimeOptions -> "Speed"
        ];

        First@AbsoluteTiming[
        len0 = Length /@ LL0;
        len1 = Length /@ LL1;
        With[
        i = Join @@ LL0,
        j = Join @@ ((0 LL0 + 1) Range[Length[LL0]])
        ,
        A0 = SparseArray[Transpose[i, j] -> 1, 26, Length[LL0]]
        ];
        With[
        i = Join @@ ((0 LL1 + 1) Range[Length[LL1]]),
        j = Join @@ LL1
        ,
        A1 = SparseArray[Transpose[i, j] -> 1, Length[LL1], 26]
        ];
        B = With[A = A1.A0,
        ArrayFlatten[

        Ramp[SparseArray[A - SparseArray[len1 - 1] Unitize[A]]],
        SparseArray[ConstantArray[1, Length[A], 1]]

        ]
        ];
        data = cf[len0, B["AdjacencyLists"]];
        result3 = Total[x^data, 2];
        ]



        0.190386




        Its result does not coincide with OP's but at least with eyorblade's one:



        result1 === result2
        result1 === result3
        result2 === result3



        False



        False



        True




        I don't know where the problem is...



        I'd like to point out that 2/3 of the computation time is wasted for doing symbolic manipulations:



        result3 = Total[x^data, 2]; // AbsoluteTiming // First



        0.127073




        The coefficient arrya can be generated much faster:



        cg = Compile[n, _Integer, l, _Integer,
        Table[n, l],
        CompilationTarget -> "C",
        RuntimeAttributes -> Listable,
        Parallelization -> True
        ];

        SetSystemOptions["SparseArrayOptions" -> "TreatRepeatedEntries" -> Total];
        First@AbsoluteTiming[
        coeffarray = With[
        i = Join @@ cg[Range[Length[data]], Length /@ data],
        j = Join @@ data
        ,
        SparseArray[Transpose[i, j] -> 1, n, 26]
        ];
        ]



        0.022717




        Testing the coefficient array:



        coeffarray.Table[x^k, k, 1, 26] == result3



        True




        Remarks



        I added the all-1-column SparseArray[ConstantArray[1, Length[A], 1]]} to B so that cf gets never fed an empty list () as second argument; compiled functions don't like that.






        share|improve this answer











        $endgroup$















          Your Answer








          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "387"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader:
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          ,
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );













          draft saved

          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmathematica.stackexchange.com%2fquestions%2f201982%2foptimising-table-wrapping-over-a-select%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown

























          4 Answers
          4






          active

          oldest

          votes








          4 Answers
          4






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          4












          $begingroup$

          Map[Total[x^Cases[L0, p:OrderlessPatternSequence[## & @@ #, ___] :> Length[p], All]] &]@L1



          2 x^4 + 2 x^5 + 2 x^6 + x^8, x^4 + x^5, 2 x^5 + x^6 + x^8,
          2 x^6 + x^8, x^4







          share|improve this answer









          $endgroup$

















            4












            $begingroup$

            Map[Total[x^Cases[L0, p:OrderlessPatternSequence[## & @@ #, ___] :> Length[p], All]] &]@L1



            2 x^4 + 2 x^5 + 2 x^6 + x^8, x^4 + x^5, 2 x^5 + x^6 + x^8,
            2 x^6 + x^8, x^4







            share|improve this answer









            $endgroup$















              4












              4








              4





              $begingroup$

              Map[Total[x^Cases[L0, p:OrderlessPatternSequence[## & @@ #, ___] :> Length[p], All]] &]@L1



              2 x^4 + 2 x^5 + 2 x^6 + x^8, x^4 + x^5, 2 x^5 + x^6 + x^8,
              2 x^6 + x^8, x^4







              share|improve this answer









              $endgroup$



              Map[Total[x^Cases[L0, p:OrderlessPatternSequence[## & @@ #, ___] :> Length[p], All]] &]@L1



              2 x^4 + 2 x^5 + 2 x^6 + x^8, x^4 + x^5, 2 x^5 + x^6 + x^8,
              2 x^6 + x^8, x^4








              share|improve this answer












              share|improve this answer



              share|improve this answer










              answered 4 hours ago









              kglrkglr

              204k10 gold badges233 silver badges463 bronze badges




              204k10 gold badges233 silver badges463 bronze badges























                  3












                  $begingroup$

                  To start off with, this has a 5-fold speed improvement on my machine. First, one helper function:



                  findIntersectionsByLength[a_, l_] := 
                  Map[If[Intersection[a, #] == a, Length[#], 0] &, l];


                  This takes a set a and searches for all members of l for which a intersect l[[i]] is a. If there is a match, then it returns the length of the match, otherwise it returns 0, for each member of l.



                  Then we tally these intersections over L0, delete all of the 0 returns, replace them with the polynomial form, and total them, for each element of L1. The Sort is included because Intersection sorts its results as well, so this ensures that it won't fail because "a","b" != "b","a".



                  Table[Total[
                  DeleteCases[
                  Tally[findIntersectionsByLength[Sort[l], L0]], 0, _] /. e_?NumericQ,
                  n_?NumericQ :> n x^e], l, L1]


                  To test this for larger samples, I generated a large sample L0 as follows:



                  alphabet = 
                  FromCharacterCode[
                  List /@ ToCharacterCode["abcdefghijklmnopqrstuvwxyz"]];
                  L0 = Table[
                  RandomSample[alphabet, RandomInteger[1, Length[Alphabet]]], i,
                  1, 1000000];


                  Using the same L1 as in the question, I get the following timings:



                  AbsoluteTiming[
                  res1 = Table[
                  Total[x^Map[Length,
                  Select[L0, SubsetQ[ToLowerCase[#1], L1[[i]]] &]]], i,
                  Length[L1]];]



                  34.0179, Null




                  AbsoluteTiming[
                  res2 = Table[
                  Total[DeleteCases[
                  Tally[findIntersectionsByLength[Sort[l], L0]], 0, _] /. e_?
                  NumericQ, n_?NumericQ :> n x^e], l, L1];]



                  6.74027, Null




                  res1 === res2



                  True







                  share|improve this answer









                  $endgroup$

















                    3












                    $begingroup$

                    To start off with, this has a 5-fold speed improvement on my machine. First, one helper function:



                    findIntersectionsByLength[a_, l_] := 
                    Map[If[Intersection[a, #] == a, Length[#], 0] &, l];


                    This takes a set a and searches for all members of l for which a intersect l[[i]] is a. If there is a match, then it returns the length of the match, otherwise it returns 0, for each member of l.



                    Then we tally these intersections over L0, delete all of the 0 returns, replace them with the polynomial form, and total them, for each element of L1. The Sort is included because Intersection sorts its results as well, so this ensures that it won't fail because "a","b" != "b","a".



                    Table[Total[
                    DeleteCases[
                    Tally[findIntersectionsByLength[Sort[l], L0]], 0, _] /. e_?NumericQ,
                    n_?NumericQ :> n x^e], l, L1]


                    To test this for larger samples, I generated a large sample L0 as follows:



                    alphabet = 
                    FromCharacterCode[
                    List /@ ToCharacterCode["abcdefghijklmnopqrstuvwxyz"]];
                    L0 = Table[
                    RandomSample[alphabet, RandomInteger[1, Length[Alphabet]]], i,
                    1, 1000000];


                    Using the same L1 as in the question, I get the following timings:



                    AbsoluteTiming[
                    res1 = Table[
                    Total[x^Map[Length,
                    Select[L0, SubsetQ[ToLowerCase[#1], L1[[i]]] &]]], i,
                    Length[L1]];]



                    34.0179, Null




                    AbsoluteTiming[
                    res2 = Table[
                    Total[DeleteCases[
                    Tally[findIntersectionsByLength[Sort[l], L0]], 0, _] /. e_?
                    NumericQ, n_?NumericQ :> n x^e], l, L1];]



                    6.74027, Null




                    res1 === res2



                    True







                    share|improve this answer









                    $endgroup$















                      3












                      3








                      3





                      $begingroup$

                      To start off with, this has a 5-fold speed improvement on my machine. First, one helper function:



                      findIntersectionsByLength[a_, l_] := 
                      Map[If[Intersection[a, #] == a, Length[#], 0] &, l];


                      This takes a set a and searches for all members of l for which a intersect l[[i]] is a. If there is a match, then it returns the length of the match, otherwise it returns 0, for each member of l.



                      Then we tally these intersections over L0, delete all of the 0 returns, replace them with the polynomial form, and total them, for each element of L1. The Sort is included because Intersection sorts its results as well, so this ensures that it won't fail because "a","b" != "b","a".



                      Table[Total[
                      DeleteCases[
                      Tally[findIntersectionsByLength[Sort[l], L0]], 0, _] /. e_?NumericQ,
                      n_?NumericQ :> n x^e], l, L1]


                      To test this for larger samples, I generated a large sample L0 as follows:



                      alphabet = 
                      FromCharacterCode[
                      List /@ ToCharacterCode["abcdefghijklmnopqrstuvwxyz"]];
                      L0 = Table[
                      RandomSample[alphabet, RandomInteger[1, Length[Alphabet]]], i,
                      1, 1000000];


                      Using the same L1 as in the question, I get the following timings:



                      AbsoluteTiming[
                      res1 = Table[
                      Total[x^Map[Length,
                      Select[L0, SubsetQ[ToLowerCase[#1], L1[[i]]] &]]], i,
                      Length[L1]];]



                      34.0179, Null




                      AbsoluteTiming[
                      res2 = Table[
                      Total[DeleteCases[
                      Tally[findIntersectionsByLength[Sort[l], L0]], 0, _] /. e_?
                      NumericQ, n_?NumericQ :> n x^e], l, L1];]



                      6.74027, Null




                      res1 === res2



                      True







                      share|improve this answer









                      $endgroup$



                      To start off with, this has a 5-fold speed improvement on my machine. First, one helper function:



                      findIntersectionsByLength[a_, l_] := 
                      Map[If[Intersection[a, #] == a, Length[#], 0] &, l];


                      This takes a set a and searches for all members of l for which a intersect l[[i]] is a. If there is a match, then it returns the length of the match, otherwise it returns 0, for each member of l.



                      Then we tally these intersections over L0, delete all of the 0 returns, replace them with the polynomial form, and total them, for each element of L1. The Sort is included because Intersection sorts its results as well, so this ensures that it won't fail because "a","b" != "b","a".



                      Table[Total[
                      DeleteCases[
                      Tally[findIntersectionsByLength[Sort[l], L0]], 0, _] /. e_?NumericQ,
                      n_?NumericQ :> n x^e], l, L1]


                      To test this for larger samples, I generated a large sample L0 as follows:



                      alphabet = 
                      FromCharacterCode[
                      List /@ ToCharacterCode["abcdefghijklmnopqrstuvwxyz"]];
                      L0 = Table[
                      RandomSample[alphabet, RandomInteger[1, Length[Alphabet]]], i,
                      1, 1000000];


                      Using the same L1 as in the question, I get the following timings:



                      AbsoluteTiming[
                      res1 = Table[
                      Total[x^Map[Length,
                      Select[L0, SubsetQ[ToLowerCase[#1], L1[[i]]] &]]], i,
                      Length[L1]];]



                      34.0179, Null




                      AbsoluteTiming[
                      res2 = Table[
                      Total[DeleteCases[
                      Tally[findIntersectionsByLength[Sort[l], L0]], 0, _] /. e_?
                      NumericQ, n_?NumericQ :> n x^e], l, L1];]



                      6.74027, Null




                      res1 === res2



                      True








                      share|improve this answer












                      share|improve this answer



                      share|improve this answer










                      answered 7 hours ago









                      eyorbleeyorble

                      6,2081 gold badge10 silver badges29 bronze badges




                      6,2081 gold badge10 silver badges29 bronze badges





















                          3












                          $begingroup$

                          A bit ugly, but also works:



                          Dot[Power[x,First[#1]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___]),#[[2]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___])]& /@ L1


                          And seems to be quite quick. For the large sample @eyorble generated I get:



                          AbsoluteTiming[Dot[Power[x,First[#1]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___]),#[[2]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___])]& /@ L1]



                          0.623584, 0, 0, 0, 0, 0




                          And for @eyorble's algorithm:




                          5.83138, 0, 0, 0, 0, 0




                          For your example we get:




                          2 x^4 + 2 x^5 + 2 x^6 + x^8, x^4 + x^5, 2 x^5 + x^6 + x^8,
                          2 x^6 + x^8, x^4




                          , as required.






                          share|improve this answer









                          $endgroup$

















                            3












                            $begingroup$

                            A bit ugly, but also works:



                            Dot[Power[x,First[#1]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___]),#[[2]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___])]& /@ L1


                            And seems to be quite quick. For the large sample @eyorble generated I get:



                            AbsoluteTiming[Dot[Power[x,First[#1]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___]),#[[2]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___])]& /@ L1]



                            0.623584, 0, 0, 0, 0, 0




                            And for @eyorble's algorithm:




                            5.83138, 0, 0, 0, 0, 0




                            For your example we get:




                            2 x^4 + 2 x^5 + 2 x^6 + x^8, x^4 + x^5, 2 x^5 + x^6 + x^8,
                            2 x^6 + x^8, x^4




                            , as required.






                            share|improve this answer









                            $endgroup$















                              3












                              3








                              3





                              $begingroup$

                              A bit ugly, but also works:



                              Dot[Power[x,First[#1]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___]),#[[2]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___])]& /@ L1


                              And seems to be quite quick. For the large sample @eyorble generated I get:



                              AbsoluteTiming[Dot[Power[x,First[#1]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___]),#[[2]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___])]& /@ L1]



                              0.623584, 0, 0, 0, 0, 0




                              And for @eyorble's algorithm:




                              5.83138, 0, 0, 0, 0, 0




                              For your example we get:




                              2 x^4 + 2 x^5 + 2 x^6 + x^8, x^4 + x^5, 2 x^5 + x^6 + x^8,
                              2 x^6 + x^8, x^4




                              , as required.






                              share|improve this answer









                              $endgroup$



                              A bit ugly, but also works:



                              Dot[Power[x,First[#1]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___]),#[[2]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___])]& /@ L1


                              And seems to be quite quick. For the large sample @eyorble generated I get:



                              AbsoluteTiming[Dot[Power[x,First[#1]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___]),#[[2]]& /@ Tally@(Length/@Cases[L0,Flatten@___,First@#,___,Last@#,___])]& /@ L1]



                              0.623584, 0, 0, 0, 0, 0




                              And for @eyorble's algorithm:




                              5.83138, 0, 0, 0, 0, 0




                              For your example we get:




                              2 x^4 + 2 x^5 + 2 x^6 + x^8, x^4 + x^5, 2 x^5 + x^6 + x^8,
                              2 x^6 + x^8, x^4




                              , as required.







                              share|improve this answer












                              share|improve this answer



                              share|improve this answer










                              answered 6 hours ago









                              amator2357amator2357

                              80710 bronze badges




                              80710 bronze badges





















                                  2












                                  $begingroup$

                                  I'd like to employ SparseArrayand thus, I convert the characters to integers in Range[1,26]. Actually, I start with integers and convert them to characters (just to be able to test against the original implementation).



                                  n = 10000;
                                  m = 100;
                                  SeedRandom[1234];
                                  LL0 = Table[RandomChoice[1 ;; 26, RandomInteger[1, 26]], m];
                                  LL1 = RandomChoice[1 ;; 26, n, 2];
                                  L0 = (FromCharacterCode[Partition[#, 1]]) & /@ (LL0 + 96);
                                  L1 = (FromCharacterCode[Partition[#, 1]]) & /@ (LL1 + 96);


                                  OP's implementation



                                  First@AbsoluteTiming[
                                  result1 =
                                  Table[Total[
                                  x^Map[Length,
                                  Select[L0, SubsetQ[ToLowerCase[#1], L1[[i]]] &]]], i,
                                  Length[L1]];
                                  ]



                                  11.3748




                                  eyorblade's implementation:



                                  findIntersectionsByLength[a_, l_] := Map[If[Intersection[a, #] == a, Length[#], 0] &, l];

                                  result2 =
                                  Table[Total[
                                  DeleteCases[
                                  Tally[findIntersectionsByLength[Sort[l], L0]], 0, _] /. e_?
                                  NumericQ, n_?NumericQ :> n x^e], l, L1]; // AbsoluteTiming



                                  2.9406




                                  My implementation:



                                  cf = Compile[len, _Integer, 1, idx, _Integer, 1,
                                  If[Length[idx] == 1,
                                  Most[0],
                                  Part[len, Most[idx]]
                                  ],
                                  CompilationTarget -> "C",
                                  RuntimeAttributes -> Listable,
                                  Parallelization -> True,
                                  RuntimeOptions -> "Speed"
                                  ];

                                  First@AbsoluteTiming[
                                  len0 = Length /@ LL0;
                                  len1 = Length /@ LL1;
                                  With[
                                  i = Join @@ LL0,
                                  j = Join @@ ((0 LL0 + 1) Range[Length[LL0]])
                                  ,
                                  A0 = SparseArray[Transpose[i, j] -> 1, 26, Length[LL0]]
                                  ];
                                  With[
                                  i = Join @@ ((0 LL1 + 1) Range[Length[LL1]]),
                                  j = Join @@ LL1
                                  ,
                                  A1 = SparseArray[Transpose[i, j] -> 1, Length[LL1], 26]
                                  ];
                                  B = With[A = A1.A0,
                                  ArrayFlatten[

                                  Ramp[SparseArray[A - SparseArray[len1 - 1] Unitize[A]]],
                                  SparseArray[ConstantArray[1, Length[A], 1]]

                                  ]
                                  ];
                                  data = cf[len0, B["AdjacencyLists"]];
                                  result3 = Total[x^data, 2];
                                  ]



                                  0.190386




                                  Its result does not coincide with OP's but at least with eyorblade's one:



                                  result1 === result2
                                  result1 === result3
                                  result2 === result3



                                  False



                                  False



                                  True




                                  I don't know where the problem is...



                                  I'd like to point out that 2/3 of the computation time is wasted for doing symbolic manipulations:



                                  result3 = Total[x^data, 2]; // AbsoluteTiming // First



                                  0.127073




                                  The coefficient arrya can be generated much faster:



                                  cg = Compile[n, _Integer, l, _Integer,
                                  Table[n, l],
                                  CompilationTarget -> "C",
                                  RuntimeAttributes -> Listable,
                                  Parallelization -> True
                                  ];

                                  SetSystemOptions["SparseArrayOptions" -> "TreatRepeatedEntries" -> Total];
                                  First@AbsoluteTiming[
                                  coeffarray = With[
                                  i = Join @@ cg[Range[Length[data]], Length /@ data],
                                  j = Join @@ data
                                  ,
                                  SparseArray[Transpose[i, j] -> 1, n, 26]
                                  ];
                                  ]



                                  0.022717




                                  Testing the coefficient array:



                                  coeffarray.Table[x^k, k, 1, 26] == result3



                                  True




                                  Remarks



                                  I added the all-1-column SparseArray[ConstantArray[1, Length[A], 1]]} to B so that cf gets never fed an empty list () as second argument; compiled functions don't like that.






                                  share|improve this answer











                                  $endgroup$

















                                    2












                                    $begingroup$

                                    I'd like to employ SparseArrayand thus, I convert the characters to integers in Range[1,26]. Actually, I start with integers and convert them to characters (just to be able to test against the original implementation).



                                    n = 10000;
                                    m = 100;
                                    SeedRandom[1234];
                                    LL0 = Table[RandomChoice[1 ;; 26, RandomInteger[1, 26]], m];
                                    LL1 = RandomChoice[1 ;; 26, n, 2];
                                    L0 = (FromCharacterCode[Partition[#, 1]]) & /@ (LL0 + 96);
                                    L1 = (FromCharacterCode[Partition[#, 1]]) & /@ (LL1 + 96);


                                    OP's implementation



                                    First@AbsoluteTiming[
                                    result1 =
                                    Table[Total[
                                    x^Map[Length,
                                    Select[L0, SubsetQ[ToLowerCase[#1], L1[[i]]] &]]], i,
                                    Length[L1]];
                                    ]



                                    11.3748




                                    eyorblade's implementation:



                                    findIntersectionsByLength[a_, l_] := Map[If[Intersection[a, #] == a, Length[#], 0] &, l];

                                    result2 =
                                    Table[Total[
                                    DeleteCases[
                                    Tally[findIntersectionsByLength[Sort[l], L0]], 0, _] /. e_?
                                    NumericQ, n_?NumericQ :> n x^e], l, L1]; // AbsoluteTiming



                                    2.9406




                                    My implementation:



                                    cf = Compile[len, _Integer, 1, idx, _Integer, 1,
                                    If[Length[idx] == 1,
                                    Most[0],
                                    Part[len, Most[idx]]
                                    ],
                                    CompilationTarget -> "C",
                                    RuntimeAttributes -> Listable,
                                    Parallelization -> True,
                                    RuntimeOptions -> "Speed"
                                    ];

                                    First@AbsoluteTiming[
                                    len0 = Length /@ LL0;
                                    len1 = Length /@ LL1;
                                    With[
                                    i = Join @@ LL0,
                                    j = Join @@ ((0 LL0 + 1) Range[Length[LL0]])
                                    ,
                                    A0 = SparseArray[Transpose[i, j] -> 1, 26, Length[LL0]]
                                    ];
                                    With[
                                    i = Join @@ ((0 LL1 + 1) Range[Length[LL1]]),
                                    j = Join @@ LL1
                                    ,
                                    A1 = SparseArray[Transpose[i, j] -> 1, Length[LL1], 26]
                                    ];
                                    B = With[A = A1.A0,
                                    ArrayFlatten[

                                    Ramp[SparseArray[A - SparseArray[len1 - 1] Unitize[A]]],
                                    SparseArray[ConstantArray[1, Length[A], 1]]

                                    ]
                                    ];
                                    data = cf[len0, B["AdjacencyLists"]];
                                    result3 = Total[x^data, 2];
                                    ]



                                    0.190386




                                    Its result does not coincide with OP's but at least with eyorblade's one:



                                    result1 === result2
                                    result1 === result3
                                    result2 === result3



                                    False



                                    False



                                    True




                                    I don't know where the problem is...



                                    I'd like to point out that 2/3 of the computation time is wasted for doing symbolic manipulations:



                                    result3 = Total[x^data, 2]; // AbsoluteTiming // First



                                    0.127073




                                    The coefficient arrya can be generated much faster:



                                    cg = Compile[n, _Integer, l, _Integer,
                                    Table[n, l],
                                    CompilationTarget -> "C",
                                    RuntimeAttributes -> Listable,
                                    Parallelization -> True
                                    ];

                                    SetSystemOptions["SparseArrayOptions" -> "TreatRepeatedEntries" -> Total];
                                    First@AbsoluteTiming[
                                    coeffarray = With[
                                    i = Join @@ cg[Range[Length[data]], Length /@ data],
                                    j = Join @@ data
                                    ,
                                    SparseArray[Transpose[i, j] -> 1, n, 26]
                                    ];
                                    ]



                                    0.022717




                                    Testing the coefficient array:



                                    coeffarray.Table[x^k, k, 1, 26] == result3



                                    True




                                    Remarks



                                    I added the all-1-column SparseArray[ConstantArray[1, Length[A], 1]]} to B so that cf gets never fed an empty list () as second argument; compiled functions don't like that.






                                    share|improve this answer











                                    $endgroup$















                                      2












                                      2








                                      2





                                      $begingroup$

                                      I'd like to employ SparseArrayand thus, I convert the characters to integers in Range[1,26]. Actually, I start with integers and convert them to characters (just to be able to test against the original implementation).



                                      n = 10000;
                                      m = 100;
                                      SeedRandom[1234];
                                      LL0 = Table[RandomChoice[1 ;; 26, RandomInteger[1, 26]], m];
                                      LL1 = RandomChoice[1 ;; 26, n, 2];
                                      L0 = (FromCharacterCode[Partition[#, 1]]) & /@ (LL0 + 96);
                                      L1 = (FromCharacterCode[Partition[#, 1]]) & /@ (LL1 + 96);


                                      OP's implementation



                                      First@AbsoluteTiming[
                                      result1 =
                                      Table[Total[
                                      x^Map[Length,
                                      Select[L0, SubsetQ[ToLowerCase[#1], L1[[i]]] &]]], i,
                                      Length[L1]];
                                      ]



                                      11.3748




                                      eyorblade's implementation:



                                      findIntersectionsByLength[a_, l_] := Map[If[Intersection[a, #] == a, Length[#], 0] &, l];

                                      result2 =
                                      Table[Total[
                                      DeleteCases[
                                      Tally[findIntersectionsByLength[Sort[l], L0]], 0, _] /. e_?
                                      NumericQ, n_?NumericQ :> n x^e], l, L1]; // AbsoluteTiming



                                      2.9406




                                      My implementation:



                                      cf = Compile[len, _Integer, 1, idx, _Integer, 1,
                                      If[Length[idx] == 1,
                                      Most[0],
                                      Part[len, Most[idx]]
                                      ],
                                      CompilationTarget -> "C",
                                      RuntimeAttributes -> Listable,
                                      Parallelization -> True,
                                      RuntimeOptions -> "Speed"
                                      ];

                                      First@AbsoluteTiming[
                                      len0 = Length /@ LL0;
                                      len1 = Length /@ LL1;
                                      With[
                                      i = Join @@ LL0,
                                      j = Join @@ ((0 LL0 + 1) Range[Length[LL0]])
                                      ,
                                      A0 = SparseArray[Transpose[i, j] -> 1, 26, Length[LL0]]
                                      ];
                                      With[
                                      i = Join @@ ((0 LL1 + 1) Range[Length[LL1]]),
                                      j = Join @@ LL1
                                      ,
                                      A1 = SparseArray[Transpose[i, j] -> 1, Length[LL1], 26]
                                      ];
                                      B = With[A = A1.A0,
                                      ArrayFlatten[

                                      Ramp[SparseArray[A - SparseArray[len1 - 1] Unitize[A]]],
                                      SparseArray[ConstantArray[1, Length[A], 1]]

                                      ]
                                      ];
                                      data = cf[len0, B["AdjacencyLists"]];
                                      result3 = Total[x^data, 2];
                                      ]



                                      0.190386




                                      Its result does not coincide with OP's but at least with eyorblade's one:



                                      result1 === result2
                                      result1 === result3
                                      result2 === result3



                                      False



                                      False



                                      True




                                      I don't know where the problem is...



                                      I'd like to point out that 2/3 of the computation time is wasted for doing symbolic manipulations:



                                      result3 = Total[x^data, 2]; // AbsoluteTiming // First



                                      0.127073




                                      The coefficient arrya can be generated much faster:



                                      cg = Compile[n, _Integer, l, _Integer,
                                      Table[n, l],
                                      CompilationTarget -> "C",
                                      RuntimeAttributes -> Listable,
                                      Parallelization -> True
                                      ];

                                      SetSystemOptions["SparseArrayOptions" -> "TreatRepeatedEntries" -> Total];
                                      First@AbsoluteTiming[
                                      coeffarray = With[
                                      i = Join @@ cg[Range[Length[data]], Length /@ data],
                                      j = Join @@ data
                                      ,
                                      SparseArray[Transpose[i, j] -> 1, n, 26]
                                      ];
                                      ]



                                      0.022717




                                      Testing the coefficient array:



                                      coeffarray.Table[x^k, k, 1, 26] == result3



                                      True




                                      Remarks



                                      I added the all-1-column SparseArray[ConstantArray[1, Length[A], 1]]} to B so that cf gets never fed an empty list () as second argument; compiled functions don't like that.






                                      share|improve this answer











                                      $endgroup$



                                      I'd like to employ SparseArrayand thus, I convert the characters to integers in Range[1,26]. Actually, I start with integers and convert them to characters (just to be able to test against the original implementation).



                                      n = 10000;
                                      m = 100;
                                      SeedRandom[1234];
                                      LL0 = Table[RandomChoice[1 ;; 26, RandomInteger[1, 26]], m];
                                      LL1 = RandomChoice[1 ;; 26, n, 2];
                                      L0 = (FromCharacterCode[Partition[#, 1]]) & /@ (LL0 + 96);
                                      L1 = (FromCharacterCode[Partition[#, 1]]) & /@ (LL1 + 96);


                                      OP's implementation



                                      First@AbsoluteTiming[
                                      result1 =
                                      Table[Total[
                                      x^Map[Length,
                                      Select[L0, SubsetQ[ToLowerCase[#1], L1[[i]]] &]]], i,
                                      Length[L1]];
                                      ]



                                      11.3748




                                      eyorblade's implementation:



                                      findIntersectionsByLength[a_, l_] := Map[If[Intersection[a, #] == a, Length[#], 0] &, l];

                                      result2 =
                                      Table[Total[
                                      DeleteCases[
                                      Tally[findIntersectionsByLength[Sort[l], L0]], 0, _] /. e_?
                                      NumericQ, n_?NumericQ :> n x^e], l, L1]; // AbsoluteTiming



                                      2.9406




                                      My implementation:



                                      cf = Compile[len, _Integer, 1, idx, _Integer, 1,
                                      If[Length[idx] == 1,
                                      Most[0],
                                      Part[len, Most[idx]]
                                      ],
                                      CompilationTarget -> "C",
                                      RuntimeAttributes -> Listable,
                                      Parallelization -> True,
                                      RuntimeOptions -> "Speed"
                                      ];

                                      First@AbsoluteTiming[
                                      len0 = Length /@ LL0;
                                      len1 = Length /@ LL1;
                                      With[
                                      i = Join @@ LL0,
                                      j = Join @@ ((0 LL0 + 1) Range[Length[LL0]])
                                      ,
                                      A0 = SparseArray[Transpose[i, j] -> 1, 26, Length[LL0]]
                                      ];
                                      With[
                                      i = Join @@ ((0 LL1 + 1) Range[Length[LL1]]),
                                      j = Join @@ LL1
                                      ,
                                      A1 = SparseArray[Transpose[i, j] -> 1, Length[LL1], 26]
                                      ];
                                      B = With[A = A1.A0,
                                      ArrayFlatten[

                                      Ramp[SparseArray[A - SparseArray[len1 - 1] Unitize[A]]],
                                      SparseArray[ConstantArray[1, Length[A], 1]]

                                      ]
                                      ];
                                      data = cf[len0, B["AdjacencyLists"]];
                                      result3 = Total[x^data, 2];
                                      ]



                                      0.190386




                                      Its result does not coincide with OP's but at least with eyorblade's one:



                                      result1 === result2
                                      result1 === result3
                                      result2 === result3



                                      False



                                      False



                                      True




                                      I don't know where the problem is...



                                      I'd like to point out that 2/3 of the computation time is wasted for doing symbolic manipulations:



                                      result3 = Total[x^data, 2]; // AbsoluteTiming // First



                                      0.127073




                                      The coefficient arrya can be generated much faster:



                                      cg = Compile[n, _Integer, l, _Integer,
                                      Table[n, l],
                                      CompilationTarget -> "C",
                                      RuntimeAttributes -> Listable,
                                      Parallelization -> True
                                      ];

                                      SetSystemOptions["SparseArrayOptions" -> "TreatRepeatedEntries" -> Total];
                                      First@AbsoluteTiming[
                                      coeffarray = With[
                                      i = Join @@ cg[Range[Length[data]], Length /@ data],
                                      j = Join @@ data
                                      ,
                                      SparseArray[Transpose[i, j] -> 1, n, 26]
                                      ];
                                      ]



                                      0.022717




                                      Testing the coefficient array:



                                      coeffarray.Table[x^k, k, 1, 26] == result3



                                      True




                                      Remarks



                                      I added the all-1-column SparseArray[ConstantArray[1, Length[A], 1]]} to B so that cf gets never fed an empty list () as second argument; compiled functions don't like that.







                                      share|improve this answer














                                      share|improve this answer



                                      share|improve this answer








                                      edited 4 hours ago

























                                      answered 5 hours ago









                                      Henrik SchumacherHenrik Schumacher

                                      64.7k5 gold badges93 silver badges179 bronze badges




                                      64.7k5 gold badges93 silver badges179 bronze badges



























                                          draft saved

                                          draft discarded
















































                                          Thanks for contributing an answer to Mathematica Stack Exchange!


                                          • Please be sure to answer the question. Provide details and share your research!

                                          But avoid


                                          • Asking for help, clarification, or responding to other answers.

                                          • Making statements based on opinion; back them up with references or personal experience.

                                          Use MathJax to format equations. MathJax reference.


                                          To learn more, see our tips on writing great answers.




                                          draft saved


                                          draft discarded














                                          StackExchange.ready(
                                          function ()
                                          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmathematica.stackexchange.com%2fquestions%2f201982%2foptimising-table-wrapping-over-a-select%23new-answer', 'question_page');

                                          );

                                          Post as a guest















                                          Required, but never shown





















































                                          Required, but never shown














                                          Required, but never shown












                                          Required, but never shown







                                          Required, but never shown

































                                          Required, but never shown














                                          Required, but never shown












                                          Required, but never shown







                                          Required, but never shown







                                          Popular posts from this blog

                                          Canceling a color specificationRandomly assigning color to Graphics3D objects?Default color for Filling in Mathematica 9Coloring specific elements of sets with a prime modified order in an array plotHow to pick a color differing significantly from the colors already in a given color list?Detection of the text colorColor numbers based on their valueCan color schemes for use with ColorData include opacity specification?My dynamic color schemes

                                          Invision Community Contents History See also References External links Navigation menuProprietaryinvisioncommunity.comIPS Community ForumsIPS Community Forumsthis blog entry"License Changes, IP.Board 3.4, and the Future""Interview -- Matt Mecham of Ibforums""CEO Invision Power Board, Matt Mecham Is a Liar, Thief!"IPB License Explanation 1.3, 1.3.1, 2.0, and 2.1ArchivedSecurity Fixes, Updates And Enhancements For IPB 1.3.1Archived"New Demo Accounts - Invision Power Services"the original"New Default Skin"the original"Invision Power Board 3.0.0 and Applications Released"the original"Archived copy"the original"Perpetual licenses being done away with""Release Notes - Invision Power Services""Introducing: IPS Community Suite 4!"Invision Community Release Notes

                                          François Viète Contents Biography Work and thought Bibliography See also Notes Further reading External links Navigation menup. 21Google Bookspp. 75–77Google BooksDe thou (from University of Saint Andrews)ArchivedGoogle BooksGoogle BooksGoogle BooksGoogle booksGoogle Bookscc-parthenay.frL'histoire universelle (fr)Universal History (en)ArchivedAdsabs.harvard.eduPagesperso-orange.frArchive.orgChikara Sasaki. Descartes' mathematical thought p.259Google BooksGoogle BooksGoogle Bookspp. 152 and onwardGoogle BooksGoogle BooksScribd.comGoogle Books1257-7979Google BooksGoogle BooksGoogle BooksGoogle BooksGoogle BooksGoogle BooksGallica.bnf.frGoogle BooksGoogle Books"François Viète"Francois Viète: Father of Modern Algebraic NotationThe Lawyer and the GamblerAbout TarporleySite de Jean-Paul GuichardL'algèbre nouvelle"About the Harmonicon"cb120511976(data)1188044800000 0001 0913 5903n82164680ola2013766880073431702w6vt1sb70287374827140948071409480